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Part  I — Elements  of 


A.  Introduction 

Binary  information  handling  is  already  so  well  known 
that  children  learn  about  in  in  grammar  school.  Indeed, 
what  could  be  simpler  than  expressing  information  in  yes-no 
or  true-false  statements  ?  Digital  computers  and  many  data 
transmission  systems  operate  in  binary  modes.  The  advan¬ 
tages  in  circuit  reliability,  simplicity  of  testing  equipment, 
and  precision  in  calculations  are  obvious.  The  principal 
disadvantage  is  the  unnecessarily  large  number  of  individual 
signal  elements  which  the  equipment  must  handle  to  perform 
a  given  task.  This  is  particularly  important  in  commercial 
communications,  where  the  customer  pays  a  fixed  amount 
per  time  unit  for  the  use  of  a  channel  of  given  bandwidth, 
such  as  an  ordinary  telephone  channel.  To  shorten  the 
t  ime  interval  of  occupying  the  channel,  the  user  is  interested 
in  transmitting  the  largest  amount  of  information  per  time 
unit.  If  .  the  information  can  be  expressed  in  numbers, 
the  user  will  learn  that  ho  needs  four  times  as  many  elements 
to  transmit  his  numbers  in  binary  elements  than  ho  would 
need  could  he  transmit  the  numbers  directly  in  decimal 
digits  (i.e.,  in  nonbinary  modes).  If  the  information  is  in 
continuous  waveforms  such  as  speech  signals  and  if  it 
has  to  be  transmitted  in  real  time,  i.e.,  without  appreciable 
delay,  the  user  will  learn  .that  the  binary  mode  will  require 
up  to  15  times  the  bandwidth  of  the  continuous  mode. 
The  communications  carrier  will  charge  higher  rates  for  the 
channel  with  the  wider  bandwidth.  In  examples  like  these, 
the  question  arises:  would  nonbinary  modes  of  operation 
be  more  economical  ?  And  if  so,  at  which  base  (number 
of  symbols  in  the  nonbinnrv  alphabet)  is  the  optimum  tradeoff 
between  the  improved  performance  of  the  nonbinary  equip- 
men t  and  its  increased  complexity  and  cost '!  It  is  the  object 
of  this  paper  to  review  the  progress  in  the  development  of 
nonbinary  systems  which  has  been  made  during  the  last 
two  decades  in  many  countries.  It  will  be  shown  that  the 
mathematical  and  graphical  tooLs  are  now  available  to  perform 
such  tradeoff  analyses. 

The  history  of  information  transmission  technology  shows 
that  nonbinary  systems  actually  preceded  binary  (Cherry  51). 
In  1823  Ronalds  used  a  single. wire  with  the  ground  as  return 
circuit,  over  which  he  transmitted  letters  of  the  alphabet 
byj  closing  the  circuit  in  one  out  of  26  subintervals  of  the 
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total  interval  reserved  for  each  character.  This  is  clearly  a 
quantized  pulse  position  modulation  (PPM)  system.  In 
1832  S.  F.  B.  Morse  introduced  his  famous  dot-dash  code 
wit  h  5  symbols  :  dot,  dash,  and  3  different  lengths  of  gaps. 
It  was  not  until  1853  that  Whitehouse  used,  for  the  first  time, 
the  highly  efficient  5-unit  telegraphy  code,  the  first  binary 
transmission  method  (Filipowsky  55). 

Nyquist  (24)  and  Hartley  (28)  most  likely  represent  the 
first  steps  toward  a  theoretical  analysis  of  nonbinary  systems 
They  defined  as  information  (in  the  notation  of  the  present 
paper)  the  expression  : 

R .vi/  =  k  .  log,  n>  ,  ( I ) 

which  indicates  that  the  information  content  of  a  word 
consisting  of  k  characters  in  a  row  (ordered  sequence)  is 
proportional  to  k  and  to  the  logarithm  of  m,  the  number  of 
symbols  in  the  alphabet  from  which  each  character  can  be 
chosen.  The  basis  of  the  logarithm  determines  the  unit  of 
information.  The  basis  2  (Hartley  28)  establishes  the  familiar 
binary  unit,  called  one  bit.  This  fundamental  relationship 
indicates  that  a  nonbinary  system  of  m  symbols  in  the  signal 
alphabet  can  transmit  log.  m  times  more  bits  per  word 
than  a  binary  system  with  the  same  number  of  elements 
per  word.  Later  investigations  by  Tuller  (49)  and  Shan¬ 
non  (48)  established  the  relations  between  the  transmission 
rate  Ra  ,  the  channel  capacity  in  the  presence  of  noise,  and 
the  entropy  of  the  information  flow.  These  relations  took 
two  important  non-deterministic  aspects  of  digital  trans¬ 
mission  systems  into  consideration  :  the  uncertainty  of 
correct  reception  of  signals  and  the  probabilistic  characteris¬ 
tics  of  the  choice  of  “  letters  ”  when  forming  “  words  ” 
for  transmitting  messages.*  Probability  theory  and  statisti¬ 
cal  mathematics  were  correctly  brought  into  the  theory  of  digi¬ 
tal  transmission  systems.  At  the  same  time  (1947)  V.  A. 
Kotel’nikov  in  Russia  submitted  his  doctoral  dissertation, 
which  was  later  translated  into  English  (Kotel’nikov  59). 
Unfortunately,  it  had  been  virtually  unknown  outside 
Russia  until  about  1956.  This  work  contains  a  special 
chapter  on  the  noise  immunity  for  signals  with  many  discrete 


*The  terms  “  letter  ”  and  “  word  ”  refer  to  any  symbol  or  group 
of  symbols,  not  necessarily  to  elements  of  written  or  printed 
languages. 
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values,  stressing  the  advantage  of  telegraphy  using  32 
orthogonal  signals — one  for  each  letter  of  the  alphabet  and 
for  some  symbols.  This  is  clearly  a  nonbinary  approach. 

All  these  pioneers  of  communications  theory  concluded 
that  tho  extraction  of  the  information-carrying  signals 
from  noise  can  best  be  performed  when  the  signals  are 
relatively  long  in  terms  of  their  reciprocal  bandwidth. 
The  more  recent  theory  speaks  of  signals  with  a  large  time- 
bandwidth  product  (also  called  TB  product  or  VVT  product) 
or  of  signals  of  high  dimensionality  (Rice  50).  If  such  large 
TB  systems  run  also  at  a  high  transmission  rate,  one  has 
only  one  approach  :  to  let  each  signal  cany  a  large  amount 
of  information.  Only  nonbinary  systems  can  achieve  this 
goal  as  evidenced  by  equation  1.  Nonbinary  systems  are 
also  more  suitable  than  binary  systems  to  fulfil  the  require¬ 
ments  for  integrated  data  systems  (Filipowsky  59).  These 
are  systems  which  combine  an  optimum  waveform  design 
with  a  number  of  additional  improvements,  such  as  error 
correction  codes,  redundancy  reduction,  and  communications 
feedback. 

The  work  of  Middleton  and  Van  Meter  (55a)  established 
the  decision -theoretical  background  for  optimum  multi- 
alternative  detection  schemes.  This  and  many  related 
papers  thereafter  lead  to  the  definition  of  mathematical 
models  for  optimum  receivers  of  nonbinary  signals.  Sec¬ 
tion  D  will  give  additional  references  about  related  analytical 
investigations. 

Since  1955,  practical  designers  of  digital  communications 
systems  have  become  interested  in  nonbinary  systems. 
Almost  400  papers  huve  been  published  about  the  theory 
and  the  practice  of  t  his  new  class  of  communications  systems. 
It.  is  the  object  of  this  review  paper  to  show  how  the  scores 
of  different  methods  can  be  grouped  and  how  closely,  in 
this  special  science,  theory  and  practice  support  each  other. 
A  particular  tool  has  been  developed  to  compare  numerically 
the  achievements  of  the  various  nonbinary  approaches. 
Called  the  utility  chart,  it.  is  an  original  contribution  by  this 
author.  Its  value  will  become  evident  to  the  reader  through¬ 
out  this  paper,  primarily  in  the  many  cases  where  practical 
measurements  on  operational  systems  will  be  compared 
with  ideal  mathematical  models. 


B.  Transmission  Problems  for  Digital  Systems 

Transmission  media  used  for  nonbinary  data  transmission 
include  practically  all  media  which  can  carry  any  type  of 
communications  signals,  most  of  all  telephone  circuits, 
high-frequency  radio  links,  scatter  links,  and  (recently) 
space  communications  links. 

Bl.  Transmitter  Limitations 

The  transmitting  equipment  of  any  data  system  generates 
t  he  information-carrying  signals,  encodes  the  input  messages 
into  these  signals,  and  matches  the  signals  to  the  constraints 
of  the  channel.  Tho  latter  task  imposes  the  principal 
limitations.  The  signal  power  must  be  limited  to  a  prescribed 
average  power  or  peak  power  and  the  power  spectrum  of 
t  he  signals  should  have  no  components  outside  the  prescribed 
transmission  band  B.  In  most  data  transmission  systems 
the  signals  are,  by  definition,  confined  to  a  fixed  time  interval, 
T,  and  the  mathematical  function  representing  the  signals 
is  assumed  to  be  identically  zero  outside  this  interval. 

Nyquist  (28)  and  Gabor  (4G)  show  that  both  the  last 
conditions  cannot  possibly  be  fulfilled  simultaneously. 
Many  recent  papers  made  the  problem  of  bandlimited  signals 
with  small  TB  the  subject  of  their  investigation.  Gibby 
and  Smith  (65)  extended  Nyquist’s  original  theory  to  trans¬ 
mission  systems  with  gradual  frequency  cutoff  and  to  band¬ 
pass  systems  with  non-ideal  filter  characteristics.  Rubin 
and  DiFraneo  (63)  developed  an  analytical  representation 
of  time-limited  signals  which  can  be  easily  applied  to  rect¬ 
angular  pulses. 

Another  important  transmitter  limitation  may  result  from 


the  sequential  combination  of  olomentary  signals.  The 
spectrum  of  a  pair  of  signals  is  usually  different  from  the 
spectrum  of  either  signal  (Bushman  63).  The  influence  of 
discontinuities  between  adjacent  elementary  signals  can  be 
of  great  importance  in  multiphase  shift  keying  (MBS K) 
systems.  Kursliin  (65)  gives  a  table  of  the  effective  TB  for 
several  sequences  of  PSK  signals. 

Most  of  the  above-mentioned  limitations  become  less 
important  if  the  signal  base  (TB)  can  be  increased.  Larger 
sets  of  longer  waveforms  can  be  used  in  nonbinary  systems 
without  reducing  the  rate  as  compared  with  equivalent 
binary  systems.  Several  such  sets  with  larger  TB  are 
discussed  in  Part  IV. 

B2.  Channel  Disturbances 

Noise  and  disturbances  created  in  the  transmission  medium 
are  tho  most  detrimental  effects  preventing  digital  data 
transmission  systems  to  achieve  the  theoretical  transmission 
rate.  The  fact  is,  and  we  shall  prove  this  point  in  section  C, 
that  most  practical  systems  do  not  oven  reach  ten  percent 
of  the  theoretical  rate.  This  fact  has  been  pointed  out  many 
times  in  tho  past  (see,  for  example,  Filipowsky  59)  and 
many  investigations  have  been  started  to  explore  the  reasons 
for  this  low  “  efficiency  ”  of  data  transmissions  systems. 
The  results  now  clearly  establish  what  many  practical  engi¬ 
neers  call  tho  “  facts  of  life  ”  :  the  mathematical  models 
which  are  used  to  represent  an  ideal  system  assume  an 
oversimplified  type  of  noise,  the  so-called  white  Gaussian 
noise  ;  and  they  also  assume  a  linear,  constant  transmission 
factor  for  tho  channel.  The  actual  observations,  as  quoted 
below,  do  not  support  this  assumption. 

The  observations  of  channel  disturbances  find  an  ap¬ 
preciable  difference  in  the  characteristics  of  at  least  throe 
different  classes  of  disturbances  :  pulsive  noiso  in  telephone 
circuits,  atmospheric  noise  in  ILF  radio  circuits,  and  man-made 
noise  in  mobile  radio  equipment.  A  kind  of  natural  distur¬ 
bance  which  follows  fairly  well  the  characteristics  of  the 
Gaussian  noise  model  apparently  is  the  noise  in  space  com¬ 
munications  circuits.  Circuit  noise  and  fluctuation  noise  in 
electronic  devices  (also  known  as  thermal  agitation  noise)  like¬ 
wise  closely  follow  the  Gaussian  model.  However,  the  im¬ 
provements  in  low  noise  solid  state  devices  reduced  the 
importance  of  these  noise  sources  when  compared  to  noise 
sources  in  the  transmission  medium. 

The  characteristics  of  pulsive  noise  in  telephone  circuits 
have  been  observed  either  directly  (Fennick  and  Nasell  66, 
Waimvright  61,  Bodonyi,  61)  or  indirectly  by  the  distribution 
of  the  errors  which  the  noise  causes  in  data  transmission 
systems  (O’Neil  65).  Special  methods  have  been  devised 
for  the  performance  of  laboratory  tests  of  data  transmission 
equipment,  using  recordings  of  real  noise  taken  from  tele¬ 
phone  circuits  (Fennick  65).  The  results  of  all  these  ob¬ 
servations,  whether  recorded  in  U.S.A.,  in  Russia  (Purtov 
et  al  65)  or  in  Europe  (Ebenau  and  Stotesbury  64)  are  in 
good  agreement.  They  indicate  that  the  noise  in  telephone 
circuits  consists  basically  of  two  components.  Ono  com¬ 
ponent,  called  the  background  noise  or  regular  component, 
can  be  represented  by  a  Gaussian  distribution,  but  its 
average  power  is  relatively  small.  The  second  irregular 
component  consists  of  high  spikes  that  occur  relatively 
infrequently,  but  with  apparently  devastating  effects  on 
short  binary  signals.  Any  noise  model  that  takes  only  tho 
average  power  of  such  noise  into  consideration  is  apparently 
inadequate. 

The  characteristics  of  atmospheric  noise  are  similar  to  the 
pulsive  noise  in  telephone  circuits.  Atmospheric  noise,  too, 
can  be  represented  by  a  non-Gaussian  component,  added 
to  a  Gaussian  component  (Shepelavey  63).  The  former 
may  be  represented,  during  short  periods  of  time  (a  few 
minutes  to  an  hour),  by  an  exponential  distribution  (Spaul¬ 
ding  64).  For  systems  below  20  MHz,  the  atmospheric noise 
is  Rayleigh  distributed  at  its  lowest  levels,  but  follows  anotli 
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exponential  law  at  higher  levels.  Results  of  an  extensive  pro¬ 
gram  of  experimental  determination  of  the  amplitude  proba¬ 
bility  distribution  (APD)  of  atmospheric  noise  have  recently 
been  published  (Spaulding  06).  The  measurements  covered 
carrier  frequencies  in  the  range  from  13  kHz  to  30  MHz 
with  channel  bandwidth  from  0.6  Hz  to  4  kHz.  Averaging 
times  from  10  minutes  to  more  than  12  hours  were  used. 
The  interesting  result  is  that  all  APD  with  observation 
parameters  covering  these  wide  ranges  in  frequency,  band¬ 
width,  and  integration  time  had  the  same  form,  anil  can  be 
represented  to  a  good  degree  of  accuracy  by  a  single  mathe¬ 
matical  description.  Despite  this  commonality  between 
various  observations  of  atmospheric  noise,  the  fine  structure 
in  the  tune  domain  may  differ  (Nakai  65)  and,  in  general, 
will  be  different  from  the  fine  structure  of  telephone  noise. 
An  additional  difference  is  duo  to  the  combined  effect  of 
atmospheric  noise  and  fading  (Conda  65,  Bello  65b),  a 
problem  unknown  in  telephone  channels. 

Man-made  noise  is  of  particular  importance  in  mobile 
radio  channels  where  neither  the  transmission  medium 
nor  the  receiving  site  can  bo  protected  against  external 
electromagnetic  influences.  It  is  of  less  importance  to  data 
transmission,  since  most  mobile  radio  links  carry  voice 
communications.  This  may  change.  Noise  created  by 
vehicles,  radar,  and  other  sources  is  already  a  significant 
problem  in  tactical  army  data  transmission  situations  (Van 
Houten  and  Rowe  63). 

Cross  talk  from  one  communications  channel  to  another 
is  a  special  kind  of  disturbance  which  may  result  from 
nonlinear  distortions  or  from  a  number  of  other  sources. 
It  may  assume  the  character  of  pulsive  disturbances  ;  indeed, 
some  part  of  telephone  noise  is  due  to  cross  talk.  In  radio 
transmission,  cross  talk  may  assume  the  nature  of  a  sinusoidal 
disturbance  and  in  electromagnetic  interference  situations  it 
may  produce  any  kind  of  well-defined  but  highly  disturbing 
waveforms.  No  special  analysis  of  cross  talk  is  advisable 
until  a  specific  design  and  its  electromagnetic  environment 
has  been  defined. 

B3.  Channel  Distortions 

It  may  be  assumed  that  the  equipment  designer  will  make 
every  effort  to  avoid  unintentional  distortions  of  the  trans¬ 
mission  signals  in  transmitter,  amplifiers  or  receiver.  This 
leaves  the  systems  engineer  still  with  a  large  variety  of 
distortions  from  the  characteristics  of  the  transmission 
medium.  These  distortions  assume  different  character  for 
at  least  four  different  transmission  media  :  lines  and  cables, 
radio  relay  links  (line-of-sight),  long  distance  radio  links 
(including  short  waves  and  all  types  of  scatter  channels) 
and  finally  space  links.  This  subdivision  parallels  to  some 
extent  the  subdivision  of  the  channel  disturbances.  Line- 
of-sighl  links  and  space  links  had  not  been  specifically 
discussed  under  channel  disturbances,  because  they  can  be 
well  represented  by  Gaussian  noise  models.  Any  other 
special  features  of  these  links  which  are  of  importance  to 
nonbinary  systems  will  be  discussed  here. 

Lines  and  cables  present  the  data  transmission  system 
designer  with  more  difficult  problems  than  the  designer  of 
telephone  voico  circuits.  It  is  primarily  the  phase  charac¬ 
teristic  of  telephone  circuits  which  is  of  negligible  importance 
to  voice  communications  but  is  of  very  high  importance 
to  data  transmission.  Phase  distortions  (also  called  delay 
distortions)  and  frequency  response  (attenuation  charac¬ 
teristics)  of  the  telephone  network  of  the  Bell  System  in 
the  United  States  are  presented  in  1960  in  a  paper  by  Alex¬ 
ander  et  al  (60),  with  additional  information  given  by 
Morris  (62).  This  can  bo  supplemented  with  information 
about  the  distortions  measured  over  Russian  (Purtov  et  al  65), 
German  (Von  Hanisch  and  Kettel  63),  and  British  (K.  L. 
Smith  et  al  62,  Croisdale  61)  telephone  and  telegraph  facilities. 
There  is  general  agreement  that  leased  lines  (Turrel  66)  are 
more  suitable  for  data  transmission  than  the  automatically 


switched  telephone  network  (Townsend  and  Watts  64). 
This  is  understandable  because  leased  lines  can  be  handpicked 
while  the  selection  of  switched  lines  is  made  at  random  by 
an  automatic  device.  It  is  also  reasonable  to  apply  some 
lengthy  effort  to  balance  anil  improve  leased  lines  by  manually 
adjusted  equalizers  while  such  action  for  switched  connections 
requires  automatically  adaptive  complex  devices.  Quite 
favourable  conditions  prevail  for  data  transmission  over 
transoceanic  cables  (Croisdale  61).  Army  facilities  (Tucker 
and  Duffy  63),  and  facilities  for  weapon  systems  (Cortizas 
and  Wolff  64)  have  to  provide  wide  safety  margins  for  tolerat¬ 
ing  distortions  to  secure  reliable  operation  under  all  opera¬ 
tional  adversities.  Such  military  facilities  impose  the  addi¬ 
tional  requirements  that  several  generations  of  older  and 
newer  equipment  have  to  be  compatible  with  each  other. 
This  requirement  makes  the  matching  and  balancing  of 
lines  rather  undesirable. 

In  addition  to  the  undesirable  delay  distortions  there 
are  three  other  peculiar  distortions  which  affect  data  trans¬ 
mission  much  more  than  voice  communications  (Morris  62). 
Drop-out,  i.e.  line  interruptions,  of  only  a  fraction  of  a  second 
may  cause  bursts  of  hundreds  of  errors  in  a  data  system, 
while  during  a  voice  communication  they  may  not  be  noticed 
at  all  or,  in  the  worst  case,  require  the  repetition  of  a  word. 
Frequency  translations  (Evans  et  al  61)  occur  primarily  in 
single  sideband  carrier  equipment  of  older  vintage.  They 
can  have  disastrous  effects  on  data  transmission  equipment. 
In  extreme  cases  it  may  happen  that  a  different  number  of 
bits  is  received  than  has  been  transmitted  or  that  the  polarity 
changes  from  “  mark  ”  to  “  space  ”  after  going  through  a 
number  of  errors.  Naturally,  protective  measures  can  be 
introduced  against  such  disturbances.  It  is,  however, 
imperative  that  the  designer  of  a  nonbinary  transmission 
system  is  fully  aware  that  such  disturbances  may  occur  in  a 
particular  facility.  Synchronization  difficulties  can  be  caused 
by  a  number  of  lino  distortions,  such  as  the  above-mentioned 
frequency  translations,  or  drop-out,  but  also  by  disturbing 
tones  caused  by  faulty  echo  suppressors  or  operators’  mistakes. 

Radio  relay  links  may  create  their  own  peculiar  distur¬ 
bances  due  to  aircraft  reflections,  weather  influences  or 
even  birds.  Many  years  of  experience  have  taught  tele¬ 
phone  companies  how  to  safeguard  the  facilities  against  most 
of  the  disturbances.  Automatic  switching  devices  and  com¬ 
plex  supervisory  systems  protect  installations  against  sudden 
failures  and  replace  disabled  units  within  seconds.  Such 
“  switchovers  ”  can  hardly  be  noticed  during  voice  conversa¬ 
tions  while  they  cause  catastrophic  errors  in  data  transmis¬ 
sions.  The  increasing  demand  for  higher  data  rates  will 
lead  to  the  provisions  of  special  wideband  channels  for  data 
transmission.  Best  candidates  are  the  12-channel  group, 
the  60-channel  supergroup  and  the  600-channol  master 
group.  Radio  relay  links  are  now  the  principal  carriers  for 
these  multiplex  facilities  and  they  surely  will  be  used  for  the 
much  simpler  data  links.  A  report  by  Suyderhoud  (65) 
describes  the  data  transmission  characteristics  of  such  wide¬ 
band  links  when  established  over  the  L-multiplex  system 
of  the  Bell  telephone  facilities  in  the  U.S.A. 

Long  distance  radio  circuits  have  been  in  use  for  telegraphy, 
amateur  services,  aircraft  communications  and  for  military 
purposes  since  the  beginning  of  this  century.  The  heavy 
disturbances  on  short  wave  links  (3  to  30  MHz)  or  on  scatter 
links  over  tropospheric,  ionospheric,  or  meteoric  scattor 
modia  make  them  a  challenge  for  data  system  designers. 
The  present  situation  of  HE  (short  wave)  data  transmission 
equipment  is  still  far  from  satisfactory,  though  great  pro¬ 
gress  has  been  made  in  the  last  10  years  (Goldberg  61,  Greim 
et  al  65,  Brennan  58,  Bello  65b)  ;  nonbinary  systems  in 
horizontally  encoded  frequency  division  multiplex  modes 
may  make  further  improvements  possible,  particularly 
when  communications  feedback  is  applied  and  when  ortho¬ 
gonal  signal  sets  are  used  (Tsikin  63).  Fritchman  anil 
Leonard  (65)  and  Fontaine  (63)  report  the  application  of 
forward  error  control  modes  to  troposcatter  data  links. 
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while  tho  Federal  Aviation  Agency  in  the  XJ.S.A.  performed 
long  distance  VHF  communications  systems  tests  (De 
Zoute  and  Pearson  64).  This  valuable  data  of  recent  field 
tests  shows  the  largo  gap  which  is  still  to  be  closed  between 
theory  and  practical  performance. 

The  longest  distances  in  the  universe  have  been  traversed 
by  data  transmission  between  planetary  deep  space  probes 
and  earth.  Such  space  communications  links  (Filipowsky  and 
Muehldorf  65a,  chapter  7)  display  special  disturbances 
which  are  normally  not  present  in  other  data  transmission 
links.  Frequency  shifts  due  to  Doppler  effect  have  been  the 
most  feared  disturbance  to  space  data  links  (Tischer  59). 
Originally  it  was  thought  that  the  best  countermeasuro 
would  be  to  frequency-modulate  the  radio  frequency  carrier 
with  a  base-band  void  of  dc  components  and  low  frequency 
components.  The  popularity  of  FM-FM  telemetry  in  the 
earlier  space  missions  is  partly  due  to  its  insensitivity  to 
Doppler  shifts.  However,  these  early  anti-Doppler  measures 
are  inefficient  in  fho  use  of  the  available  transmitter  power 
and  bandwidth.  The  requirement  for  precise  tracking  of 
spacecraft  is  another  important  reason  for  improved  anti- 
Doppler  measures.  Digital  correlation  detection  of  Dopplor 
shifted  signals  (Miller  64)  is  now  generally  applied  for  acquisi¬ 
tion  and  locking-in  signals  from  deep  space  probes.  Tho 
method  provides  local  oscillator  signals  in  the  Earth-based 
receivers,  which  track  the  Doppler  shifted  variable  carrier 
frequency  with  the  result  that  the  intermediate  frequency 
in  the  receiver  is  basically  free  from  Dopplor  variations 
(Mathison  62). 

All  signals  between  spacecraft  and  Earth  have  to  pene¬ 
trate  the  Earth’s  atmosphere  and  distortions  may  occur  on 
such  paths.  The  nature  and  the  intensity  of  these  distor¬ 
tions  and  disturbances  depend  on  the  frequency  range 
(Tischer  65).  Nonbinary  data  systems  with  their  more 
complex  waveforms  need  careful  attention  to  these  propaga¬ 
tion  phenomena  (Baghdady  65).  Attenuation,  polarization, 
and  other  effects  of  the  interplanetary  medium  on  S-band 
links  are  discussed  by  Easterling  and  Goldstein  (65).  Surveys 
of  the  effects  of  propagation  effects  on  space  data  links  are 
also  available  in  a  recent  NASA  report  (Dabul  66)  and  in  a 
Russian  book  on  space  radio  communications  in  English 
translation  (Petrovich  and  Kamnev  66). 

Summarizing  this  section  one  can  state  that  much  has  been 
learned  in  tho  last  decade  about  the  adverse  effects  of  trans¬ 
mission  channels  on  data  transmission  systems.  Unfor¬ 
tunately,  the  conclusion  is  that  the  characteristics  differ 
greatly  in  different  media  and,  even  in  channels  of  the  same 
type,  the  adverse  effects  vary  with  geographical  location, 
season,  daytime,  and  many  other  environmental  factors. 
Data  systems  will  need  high  flexibility  and,  most  likely, 
adaptive  modes  of  operation  to  meet  those  largo  variations 
in  channel  characteristics.  Theoretical  and  practical  indica¬ 
tions  are  that  nonbinary  systems  offer  these  desirable 
characteristics. 

C.  Performance  Criteria 

The  merits  and  costs  of  several  competing  data  systems 
can  be  compared  only  after  performance  criteria  have  been 
established.  These  criteria  should  be  equally  suitable  for 
theoretical  analysis,  for  laboratory  measurements  under 
simulated  channel  conditions,  and  for  field  tests  under 
actual  operational  conditions. 

Cl.  Definitions 

There  are  at  least  six  considerably  different  kinds  of 
characteristics  which  have  to  be  considered  when  judging 
tho  overall  performance  of  any  data  system. 

Tho  first  characteristic  is  a  quantity  indicating  the  achieve¬ 
ment  or  the  result  of  a  data  transmission  operation,  tho  trans¬ 
mission  rale  (R)  or  tho  amount  of  information  transmitted 
per  unit  tune,  usually  expressed  in  bits  per  second  (Rn). 


Some  publications  prefer  the  natural  logarithm  or  the  decimal 
logarithm  in  place  of  the  logarithm  of  baso  two.  Correspond¬ 
ingly,  the  rate  may  also  be  expressed  in  nits  per  second  or 
in  dits  per  second.  In  systems  with  time-variable  rate, 
an  average  transmission  rate  (R)  must  be  defined  and  the 
averaging  interval  should  be  indicated. 

Tho  second  characteristic  is  a  quantity  indicating  tho 
quality  of  the  achievement.  This  is  usually  the  error  ratio  (e), 
i.e.,  the  ratio  of  the  numbor  of  digits  received  in  error  to  tho 
total  number  of  digits  received.  This  ratio  should  bo  deter¬ 
mined  at  tho  output  of  the  system.  It  is  advisable  to 
express  this  output  error  ratio  as  a  ratio  of  tho  number  of 
bits  in  error  to  the  total  number  of  bits  delivered  to  the 
user,  although  some  authors  prefer  to  define  tho  output 
error  as  the  ratio  of  the  number  of  bits  in  error  to  the  numbor 
of  bits  correctly  received  by  the  user.  Both  definitions  of 
the  bit  error  ratio  (ojj)  asymptotically  approach  each  other 
when  the  number  of  errors  approaches  zero.  However, 
it  is  very  important  to  notice  that  many  systems  are  deliver¬ 
ing  tho  output  information  to  tho  user  in  nonbinary  form. 
An  example  is  the  characters  of  the  alphabot  printed  by  a 
teletypewriter.  In  such  cases  it  is  necessary  to  measure 
the  character  error  ratio  (ec)  and  to  calculate  tho  equivalent 
bit  error  ratio  (ocn)-  The  conversion  equation  between 
these  two  magnitudes  is  to  be  determined  in  line  with  the 
design  of  tho  decoder.  If  no  information  about  the  nature 
of  the  character  error  is  delivered  to  the  user,  tho  equivalent 
bit  error  ratio  will  be  highest.  If  full  information  can  bo 
forwarded  to  tho  user,  such  as  indicating  the  second  and 
third  most  likely  received  character  along  with  the  most 
likoly  received  character,  the  equivalent  bit  error  ratio  will 
be  smallest. 

The  upper  and  lower  bounds  for  the  equivalent  bit  error 
ratio  when  the  character  error  ratio  is  measured  arc  as 
follows  : 

ec/n  <  eEB  <  ec/2  ,  (2) 

where  n  is  the  number  of  bits  carried  by  one  character.  If 
all  m,  characters  of  an  alphabet  are  used  equally  frequently, 
equation  3  applies  : 

n  =  logj  n,  (3) 

The  reader  is  referred  to  the  following  publications  for 
additional  information  :  Okunev  (64),  Servinskiy  (64), 
Beinhocker  (61),  Cohn  (63),  Filipowsky  and  Muehldorf 
(65a,  Appendix  B),  Silverman  and  Balser  (54),  Haekett  (63), 
Posner  (64). 

Tho  third  and  fourth  characteristics  are  two  quantities 
which  indirectly  determine  the  dollar  cost  of  tho  system  and 
its  equipment.  They  are  the  bandwidth  of  the  transmission 
channel  and  the  signal  power  of  the  transmitter  at  the  input 
to  tho  channel.  The  bandwidth  (B)  is  in  some  cases  completely 
at  the  discretion  of  the  designer  (certain  space  channels)  ; 
in  other  cases,  it  is  an  absolutely  fixed  constant  given  to  the 
designer  (telephone  channels).  The  actually  available  band¬ 
width  rarely  can  bo  completely  defined  by  a  single  number 
(in  Hz).  In  some  cases  it  is  defined  by  regulatory  bodies 
(Federal  Communications  Commission  in  the  U.S.A.)  ;  in 
other  cases,  by  the  characteristics  of  the  transmission  medium 
and  amplifying  equipment  (frequency  response  characteristics 
of  telephone  circuits)  ;  and,  in  still  other  cases,  by  the  charac¬ 
teristics  of  the  receiving  equipment  (filters  in  phase-lock 
loops,  Karras  65).  This  situation  forces  designer's  to  arrive 
at  an  effective  bandwidth  (B  „//)  when  attempting  to  compare 
nonbinary  systems  which  operate  over  channels  of  different 
bandwidth.  The  effective  bandwidth  can  be  easily  derived 
from  a  frequency  response  curve  of  the  channel  (Karras  65). 

It  is  also  important  to  recognize  that  many  systems  perform 
bandwidth  translations  through  which  they  either  spread 
out  the  energy  over  a  much  wider  band  (band  spread  system) 
or  concentrate  it  into  a  much  smaller  band,  when  going 
from  the  input  signals  (base-band)  to  the  transmission 
signals  (carrier  band).  In  these  cases  one  must  discriminate 
between  the  bandwidth  of  the  information-carrying  inp 
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signals  at  tho  transmitter  (Bin),  the  bandwidth  of  the  trans¬ 
mission  channel  (B/r)  and  the  bandwidth  of  the  information- 
carrying  output  signals  (Bom)  (Filipowsky  and  Muehldorf  65a, 
section  4.1.4).  For  the  purpose  of  this  paper,  we  assume 
that  the  input  information  and  the  output  information  are 
in  binary  form,  and  we  arc  concerned  only  with  the  trans¬ 
mission  bandwidth  B (r.  The  notation  B  will  always  be 
understood  as  tho  effective  bandwidth  in  the  transmission 
channel  unless  stated  differently. 

The  signal  power  S  is  the  second  parameter  which  is  partially 
at  the  discretion  of  the  designer.  Naturally,  there  arc 
constraints  to  an  unlimited  increase  of  the  signal  power. 

One  such  constraint  in  telephone  circuits  is  the  maximum 
voltage  or,  current  for  which  the  components  and  lines  are 
rated.  Another  constraint  is  tho  danger  of  cross  talk 
between  circuits  if  the  power  that  is  used  in  some  of  the  links 
is  too  large.  Particularly  severe  limitations  apply  to  circuits 
going  over  carrier  systems  because  these  systems  are  designed 
with  carrier-suppressed  speech  circuits  in  mind  (Thiess  60). 
This  means  that,  the  frequency  divided  (FD)  multiplex 
equipment  is  loaded  only  during  periods  of  active  speech 
and,  of  course,  only  a  small  percentage  of  the  total  time  used 
for  a  telephone  conversation  is  filled  with  active  speech 
(Holbrook  and  Dickson  39).  If  many  circuits  of  an  FD 
multiplex  system  are  simultaneously  loaded  with  data 
channels,  Jan  overload  situation  may  result  which  will  give 
less  power  to  the  individual  circuits  than  assumed  in  tho 
design  calculations  and  which  may  also  cause  undue  dis¬ 
tortions. 

in  radio  circuits  the  power  constraint  is  in  the  capability 
of  the  output  amplifier  devices,  which  are  either  average 
power  limited  or  peak  power  limited.  Specially  designed 
high-pulse  transmitters  can  achieve  particularly  largo 
peak  power  to  average  power  ratios  and  thus  are  particularly 
suitable  for  certain  pulsivo  systems. 

The  notation,  S.  will  designate  the  average  power  of  the 
information-carrying  signals  when  integrated  over  a  duration 
of  many  signal  intervals.  For  many  calculations  it  is  actually 
more  important  to  use  the  signal  energy  in  place  of  the  signal 
power.  It  is  defined  ns  : 

r'1  „ 

Ej  =  f,-(t)  dt  =  S,  .  (t,  -  t,)  (4) 

J  Ij 

K  is  the  average  value  over  a  large  number  of  energy 
expressions  of  signal  symbols  or  signal  characters.  Wo 
particularly  use  the  expression  Ez>  to  define  the  average 
signal  energy  of  one  digit  ;  i.e.,  of  one  symbol  out  of  the 
alphabet  of  m  different  symbols.  In  many  cases  all  symbols 
of  the  alphabet  will  have  the  same  energy,  but  there  are 
exceptions,  such  as  multilevel  systems.  A  most  important 
magnit  ude  is  EB  ,  the  signal  energy  per  bit  of  output  informa¬ 
tion.  Considering  the  definition  of  S  and  of  It,  one  arrives 
at  the  relationship  : 

Eb  =  —  =  S  .  Tc  (5) 

in  watt -seconds,  with  Tb  being  defined  as  the  bit  period  in 
seconds. 

The  unique  magnitude  for  tho  characterization  of  white 
noise  is  the  noise  power  density  (No),  which  is  defined  as  the 
average  noise  power  (N)  per  unit  bandwidth  :* 


N„ 


N 

B'r; 


(6) 


in  watt-seconds. 


*“  White  ”  noise  is  defined  as  noise  with  a  uniform  power 
spectrum,  i.e.,  with  N„  const  ,  (independent  of  frequency).  In 
some  theoretical  investigations  the  p'oWer  spectrum  is  plotted 
for  positive  and  negative  frequencies.  Wo  refer  X„  to  the  one¬ 
sided  power  spectrum,  avoiding  a  factor  2  in  the  denominator 
of  equation  0. 


Noise  is  produced  at  many  places  in  the  channel  and  in  the 
receiver.  The  most  sensitive  spot,  where  noise  can  enter  a 
data  link  Ls  a  place  which  is  followed  by  high  gain  amplifica¬ 
tion,  i.e.,  the  input  to  the  final  amplifier  or  radio  receiver. 
It  is  normally  this  place  to  which  all  other  noise  sources 
are  referred  and  their  total  action  is  then  expressed  as  the 
average  operational  noise  temperature  (Top)  (Filipowsky 
and  Muehldorf  65a,  pp.  176-247).  Jf  this  magnitude  is 
supplied,  then  N,  can  be  calculated  from  : 

N„  =  K bo  •  Top  (7) 

in  watt-seconds.  Kbo  is  the  Boltzman’s  constant  1.38044 
x  10-  =  ’ VAS/CK. 


These  simple  relationships  of  equations  6  and  7  are  not 
sufficient  to  describe  more  complex  noise  models.  At  this 
time,  however,  practically  all  mathematical  models  of 
nonbinary  systems  have  been  developed  with  white  Gaussian 
noise  as  the  only  disturbance.  Though  the  author  is  com¬ 
pletely  aware  of  the  insufficient  approximation  to  real  life 
channels,  which  these  models  offer,  he  is  also  convinced 
that  the  white  Gaussian  noise  model  is  an  interesting  and 
mathematically  easily  accessible  basis  for  a  first  comparison 
of  the  many  nonbinary  modes  of  operation.  It.  will,  therefore, 
bo  used  as  the  basis  of  the  trade-off  charts  to  be  discussed 
in  the  next  section.  Whenever  appropriate,  comments  will 
be  inserted  to  indicate  the  possibilities,  and  also  the  weak¬ 
nesses  of  any  comparison  of  actual  field  tests  with  these 
highly  idealized  mathematical  models.  Ten  to  twenty 
years  from  now,  engineers  will  have  highly  improved  mathe¬ 
matical  models  of  special  channels  :  performance  predictions 
will  be  possible  with  adequate  accuracy  :  and  computer 
simulations  will  permit  fast  experimentation  within  wide 
ranges  of  parameters,  without  ever  building  any  of  the  systems 
under  consideration.  But  the  students  will  still  have  to 
use  the  simplified  models  which  we  are  using  here,  before 
they  will  be  able  to  understand  the  highly  sophisticated 
more  accurate  models. 

Basing  tho  present  considerations  on  the  white  noise 
model  with  Gaussian  amplitude  distribution,  we  first  notice 
how  important  it  is  to  measure  or  to  specify  how  much  the 
signal  stands  out  of  the  noise.  The  quantity  historically 
used  for  this  purpose  is  the  signal-to-noise  power  ratio  (SNR), 
i.o.,  the  power  advantage  which  the  signal  can  claim  above 
the  noise.  This  term,  also  called  tho  power  contrast,  will 
l)e  symbolized  by  the  Greek  lett  er  a  : 


a 


in  numbers. 


S  S 
N  N„  IV 


S 


lx  no  I  „j>B|r 


(«) 


There  are  many  different  locations  in  a  receiver  where 
the  power  contrast  can  be  measured.  Section  C3  will 
present  a  short  review  of  the  important  signal  processing 
characteristic,  which  is  nothing  else  but  the  graphical 
presentation  of  the  output  SNR  ( a0,n )  as  a  function  of  the 
input  SNR  ( Hi,,)  of  a  nonlinear  signal  processing  device 
(Filipowsky  and  Muehldorf  65a,  pp.  176-247). 

Closely  related  1o  the  SNR  is  a  magnitude  (p)  defined  as  the 
average  signal  power  over  the  noise  power  density  : 


in  Hz. 


S.B(r 

N 


a  .  B,r 


('•») 


This  quantity  has  the  dimension  of  a  frequency  (!■>-')  and 
ii  is  something  of  a  characteristic  rale  of  the  system,  also 
called  “  signal  contrast  frequency  ”  (Filipowsky  and  Muehl¬ 
dorf  65a,  pp.  176-247).  Equation  9  shows  that  it  is  the 
product  of  the  transmission  bandwidth  and  tho  power 
contrast.  One  can  see  that  a  system  with  a  power  contrast 
of  10  (10  db)  has  a  characteristic  rate  of  10  times  the  trans- 
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mission  bandwidth.  From  this,  one  concludes  that  the 
ratio  between  the  actually  achieved  rate  and  the  characteristic 
rate  will  be  something  like  a  measure  of  the  communications 
efficiency  of  a  system. 

The  difficulty  when  operating  with  p,  the  characteristic 
rate,  is  the  fact  that  it  is  not  a  normalized  magnitude  like  a. 
which  is  a  pure  number  and  which  may  be  conveniently 
expressed  in  decibels,  a  logarithmic  measure.  Many  authors 
used,  therefore,  another  dimensionless  magnitude  which  has 
been  called  the  energy  contrast,  or  E  over  No  (Van  Horn  61, 
Muohldorf  et  al  59).  We  shall  use  the  Greek  letter  e  for  this 
important  quantity,  which,  under  the  previously  mentioned 
restraints,  is  defined  by  : 


E  _  S  .T 
N0~ 


a  .  B,r .  T 


(10) 


in  numbers. 

The  energy  contrast  can  be  defined  for  one  digit  («o  ,  E n  , 
Td),  for  one  bit  ( („  ,  Eb,  Tb),  or  for  any  other  convenient 
signal  energy  and  time  unit.  Notice  thut,  N  „  also  has  the 
dimension  of  energy  (power  over  bandwidth).  The  bit  energy 
contrast  en  is,  therefore,  a  ratio  of  the  energy  which  the 
designer  is  willing  to  spend  for  transmitting  one  bit  of 
information  to  the  energy  of  the  adverse  effects  (N„).  It 
will  be  shown  in  section  C2  that  the  ideal  communications 
system  postulated  by  Shannon  as  the  upper  bound  of  the 
channel  capacity  needs  exactly  one  unit  of  en  to  transmit 
without  errors  one  bit  of  information  per  second  in  a  channel 
of  1  Hz  bandwidth. 

The  sixth  and  last  quantity  of  importance  for  the  evalua¬ 
tion  of  nonbinary  data  system  is  a  measure  of  the  distortions 
which  the  information-carrying  signals  will  suffer  between 
transmitter  and  receiver.  In  section  B3  a  large  variety  of 
distortions  was  listed  which  may  affect  the  various  trans¬ 
mission  media.  Apparently  no  single  number  can  bo  found 
which  could  serve  as  an  adequate  measure  for  characterizing 
all  these  effects.  For  the  purposo  of  this  paper  wo  shall 
assume  that  all  distortions  may  be  negligible  (linear  trans¬ 
mission  channel). 


C2.  The  Utility  Chart 

The  six  basic  characteristics,  discussed  above,  will  now 
be  used  in  a  practical  design  chart. 

(1)  Transmission  rate  (Kb) 

(2)  Error  ratio  (e) 

(3)  Transmission  bandwidth  (B<r) 

(4)  Signal  power  (S) 

(5)  Noise  power  (N) 

(0)  Distortion  measures 

No  practical  chart  should  deal  with  six  variables  at  the 
same  time.  Design  engineers  need  normalized  measures  ; 
i.e.,  a  reduction  of  the  number  of  variables  by  defining 
relative  magnitudes.  They  also  prefer  dimensionless  quan¬ 
tities,  such  as  efficiency,  density,  or  other  “  percentage  ” 
measures.  Many  attempts  have  been  made  in  the  past  to 
arrive  at  such  design  charts  for  various  modulation  methods 
(Jelonck  52,  Helstrom  60,  Sanders  59b,  60a).  The  chart 
used  in  all  the  following  sections  is  a  modification  of  several 
of  these  earlier  attempts.  It  has  demonstrated  its  practical 
value  in  our  laboratories.  The  following  logical  considera¬ 
tions  lead  automatically  to  this  chart. 

The  first  step  is  to  eliminate  all  second-order  considerations. 
Thus  distortion  measures  arc  not  considered  during  the 
first  approach.  We  assume  linear  channels.  Distortion 
measures  can  always  bo  introduced  as  fixed  parameters. 

The  second  step  is  to  simplify  any  quantities  that  cannot 
be  expressed  in  simple  terms,  by  making  restricting  assump¬ 
tions.  This  has  been  indicated  in  section  B1  by  using  N„ 
as  a  single  number  representing  the  most  regular  kind  of 


noise  and  by  introducing  an  effective  transmission  band¬ 
width,  Bi  r . 

The  third  step  is  to  identify  magnitudes  which  serve  merely 
as  limiting  bounds  and  plot  any  charts  for  fixed  parameter 
values  of  such  magnitudes.  This  can  be  done  with  the  error 
ratio.  Most  of  our  curves  have  been  calculated  for  a  fixed 
bit  error  ratio  of  10" 3  (i.o.,  for  one  bit  in  error  out  of  1,000 
bits  of  information  received  at  the  output  of  a  link). 

After  these  simplifying  steps,  one  must  still  deal  with  four 
important  quantities  :  R,  Bjr ,  S,  and  N. 

The  fourth  step  is  to  normalize  by  using  ratios  of  these 
magnitudes.  These  ratios  should  be  dimensionless.  There 
are  only  two  physical  dimensions  attached  to  those  four 
magnitudes.  Rb  and  B(r  have  the  dimension  sec-1  and  S 
and  N  have  the  dimension  watts.  Thus  we  define  a  new 
magnitude  : 

Db=Rb/B1(.  (11) 

in  numbers. 

This  magnitude  may  be  called  the  information  density  or, 
when  Rb  is  expressed  in  bits  per  second,  the  bit  density. 
It  is  defined  as  the  rate  in  bits  per  second  over  the  trans¬ 
mission  bandwidth  in  cycles  per  second.  For  example,  a 
bit  density  of  one  bit  per  second  per  Hz  (i.o.,  a  bit  density  of 
one)  will  permit  a  system  to  send  3000  bits  per  second  over  a 
channel  of  3000  Hz.  It  is  convenient  to  express  D«  in. 
decibels  following  the  equation  : 

D™  =  10[log10  Ra  —  log10  B„]  (12) 

in  decibels. 

In  this  logarithmic  measure,  negative  db  numbers  will 
indicate  a  rate  less  than  the  bandwidth  and  a  positive 
number  will  indicate  a  rate  higher  than  the  bandwidth. 
If  the  rate  in  bits  per  second  is  equal  to  the  bandwidth  in 
Hz,  then  the  bit  density  is  zero  decibel. 

A  ratio  of  the  other  two  remaining  magnitudes,  S  and  N, 
has  already  been  defined  in  equation  8  as  the  power  contrast 
(<?,  power  SNR).  It  would,  therefore,  be  convenient  to 
plot  a  as  a  function  of  D  or  vice  versa,  and  to  declare  that 
system  the  better  one  which  requires  the  smaller  power 
contrast  for  achieving  the  same  bit  density  or  which  achieves 
a  higher  bit  density  for  the  same  power  contrast.  Indeed, 
this  normalization  has  been  used  many  times  in  the  past 
for  comparing  binary  systems  with  sampled  or  analog 
communications  systems.  Nonbinary  systems,  however, 
may  be  used  with  equal  advantage  in  ranges  of  very  small 
power  contrast  and  also  of  very  high  power  contrast.  In 
both  theso  cases  they  will  operate  at  about  the  same  energy 
contrast.  It  is,  therefore  more  advantageous  to  use  a 
presentation  which  plots  the  bit  density  as  a  function  of  the 
energy  contrast.  A  chart  of  this  nature  was  first  introduced 
by  Sanders  (60a).  The  version  introduced  here  has  an 
additional  advantage  over  Sanders’  chart  in  that  both  axes 
increase  in  the  normal  way  from  left  to  right  and  from  bottom 
to  top  when  the  corresponding  magnitudes  improve  in  the 
desirable  direction. 

We  define,  similarly  as  Sanders,  a  kind  of  communications 
efficiency.  We  call  it  the  utility  (u)  of  the  system.  It  is 
defined  as  the  ratio  of  the  achievement  (the  bit  density  D) 
to  the  effort  to  reach  this  achievement  (the  power  contrast  a)  : 
For  the  ideal  system  the  utility  assumes  the  value  1  when  the 
signal  power  and  noise  power  are  equal.  If  a  practical 
system  needs  a  signal  power  larger  t  han  noise  power,  it  has  a 
utility  smaller  than  one.  Thus  the  utility  may  be  expressed 
in  percentage  values,  quite  similar  to  the  efficiency  in  mechani¬ 
cal  engineering.  The  utility  is  related  to  the  other  magni¬ 
tudes  in  the  following  way  : 


u  Db  _  Rb_  N  =  R  N,  Rb 

(7  ByB  S  S  p 


(13) 


Da  1  =  N.  =  J_ 

a  °  Tb  .  S  Eb  £b 


(14) 
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Db 


Kfl  l  _  l  1 
Br/i  a  Tb  .  Btb  <7 


(15) 


in  numbers. 

Equations  13,  14,  15  show  many  ways  of  relating  the 
utility  to  the  fundamental  magnitudes  defined  in  section  Cl. 
Equation  13  shows  that  the  utility  is  also  the  ratio  of  the 
actually  achieved  transmission  rate  Rb  to  the  characteristic 
rate  of  the  system.  Equation  14  shows  the  very  important 
fact  that  the  utility  is  nothing  but  the  reciprocal  value  of 
tho  energy  contrast.  Equation  18  shows  that  the  utility 
can  be  kept  constant  when  trading  off  signal  power  against 
the  so-called  time-bandwidth  product,  Tb  B rn  ■  For  simpler 
notation  the  letter  j8  is  introduced  to  characterize  the  time- 
bandwidth  product,  frequently  also  called  the  signal  base. 

/J  =  T  .  Bra  (16) 


The  signal  base  may  be  taken  for  a  bit  of  transmitted 
information  (fin),  for  a  digit  (fin),  or  a  whole  message  (Pm). 
In  each  case  one  must  select  the  correct  value  for  the  signal 
duration  (Tb  ,  TD  ,  T.u).  Tho  signal  base,  like  most  of  the 
other  characteristic  magnitudes,  is  dimensionless. 

The  importance  of  choosing  this  ut  ility  as  the  key  figure-of- 
merit  of  data  systems  can  still  be  better  recognized  when 
plotting  the  utility  of  Shannon’s  ideal  and  hypothetical 
error-free  communications  system  in  a  logarithmic  scale  in 
decibels.  This  is  done  in  fig.  1.  Shannon’s  upper  bound  of 
the  channel  capacity  of  this  ideal  system  for  an  average 
power  limited  transmitter  is  usually  written  in  the  following 
form  (Shannon  48)  : 

C  =  B.T.log2(l  .  (17) 

C  is  the  maximum  number  of  bits  that  the  ideal  system 
can  transmit  in  the  time  (T)  without  error.  Introducing 
the  notations  of  this  paper  : 

Rb  =  —  =  Bya  log™  (1  +  a)  (18) 


Rb 
B  TR 


=  Db  =  logjj  (I  -1-  a) 


D„ 


Db 


(19) 


10  Logj q  u  =  10  log,,,  Db  -  10  log10  (2°b  -  1)  (20) 

u^  _  D<ib  -  10  log iq  (2°B  -  1)  (21) 

B 

uai,  _  djs  _  3.01  .  Dfl  for  >  io  (22) 


U<a  =  -10  log,„  [0.6931  +  0.2404  .  Db  + 

+  0.0555  .  Db +  ...  ]  for  Db  <0.1  (23) 

Equation  21  is  the  equation  for  the  utility  curve  of  tho 
ideal  system,  which  is  plotted  in  fig.  1  as  curve  1.  Its  upper 
side  is  shaded  to  indicate  that  this  curve  represents  an 
upper  bound  for  all  other  systems  under  the  stated  assump¬ 
tions  of  white  Gaussian  noise  and  average  signal  power 
limitations.  The  interesting  fact  is  that  this  system  leaves 
complete  freedom  to  operate  at  large  bit  densities  (right  side 
of  fig.  1)  or  small  bit  densities  (left  side).  It  also  shows  that 
the  ideal  svstem  has  the  utility  1  (0  db)  for  a  bit  density 
of  1  (0  db). 

There  is  an  approximate  expression  for  small  values  of 
Db  (equation  23),  which  indicates  that  tho  utility  for  tho 
ideal  system  can  exceed  the  value  1,  but  that  the  highest 
utility  ever  to  be  reached  is  —10  log,  „  0.6931  =  1.592  db. 
This  value  has  been  frequently  quoted  in  the  literature  as 
(log,  2)->  =  1.441  in  numbers. 

Another  approximation,  given  in  equation  22,  is  valid 
for  bit  densities  above  10.  Notice  that  Db  in  the  first  term 
is  expressed  in  decibels  but,  in  the  second  term,  it  is  expressed 
in  numbers. 


Figure  1  .—The  utility  chart  for  the  ideal  communications  system 
(Shannon  1948)  and  for  a  number  of  practical  data  systems. 
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Figure  1  shows,  in  addition  to  the  curves  for  the  ideal 
systems,  the  location  of  a  number  of  practical  systems 
in  the  same  chart.  This  should  demonstrate  the  valuo  of  the 
chart  as  a  means  for  comparing  the  achievement  of  practical 
systems  with  their  theoretical  possibilities,  but  it  should 
also  indicate  the  large  range  for  improvement  that  is  still 
available.  Most  practical  systems  operate  10  to  20  db 
below  their  theoretical  utility  ;  i.e.,  they  operate  with  less 
than  10  percent  efficiency.  The  dashed  diagonal  lines  in 
fig.  1  represent  linos  of  constant  power  contrast.  The  line 
for  zero  db  power  contrast  goes  through  the  origin  of  the 
chart  where  it  meets  the  ideal  system. 

The  best  way  to  understand  the  value  of  the  utility  chart 
is  to  start  with  a  hypothetical  design  at  the  most  inefficient 
location  (the  left  lower  corner)  and  to  attempt  an  improve¬ 
ment  in  efficiency.  The  goal  is  the  ideal  system,  line  1. 
There  are  three  fundamentally  different  approaches. 

(1)  The  power -limited  design  :  One  tries  to  increase  the 
utility  by  reducing  the  power  which  tlio  transmitter 
lias  to  supply,  while  keeping  the  transmission  rate 
and  bandwidth  constant  (vertical  line  going  upwards). 

(2)  The  bandwidth-limited  design  :  Here  the  opposite  is 
the  case.  The  bandwidth  is  prescribed  (telephone 
channel)  but  the  power  may  be  increased  or  the  noise 
may  be  reduced  by  design  improvements.  The 
principal  direction  of  the  design  improvement  is  to 
the  right. 

(3)  The  cost-limited  design  may  bo  followed  whon  both 
bandwidth  and  power  are  at  the  discretion  of  the 
designer  or  if  he  is  limited  in  both  values  to  some 
extent.  This  dosign  follows  in  general  a  constant  a  line. 

The  individual  lines  and  points  in  fig.  1,  which  are  markod 
by  capital  letters,  represent  the  practical  systems  discussed 
below. 

(1)  Operating  points  ABC  :  A  teletypewriter  system 
tested  in  the  United  Kingdom  in  laboratory  tests  with 
Gaussian  noise  generated  by  a  noise  generator  (Rosie  63). 
The  purpose  of  those  tests  was  to  demonstrate  the 
advantages  of  higher  order  detection  of  complete 
teletypewriter  characters  as  compared  with  a  bit-by-bit 
detection  of  the  individual  elements  of  a  character. 

Point  B  is  the  theoretical  value  for  an  error  ratio 
of  10“1 2  3.  This  point  shows  how  well  a  design  could 
operate  in  the  best  case.  It  would  use  an  alphabet 
of  32  waveforms,  each  consisting  of  5  binary  elements 
(20  ms  long).  This  theoretical  model  assumes  that  a 
correlation  detector  and  maximum  likelihood  decider 
(MLD)  will  be  applied. 

Point  C  is  the  operating  point  for  a  standard  bit-by- 
bit  teletypewriter  receiver.  It  loses  utility  of  approxi¬ 
mately  5  db  due  to  the  sub-optimal  detection  method 
of  point  C.  This  point  results  from  measurements 
in  the  laboratory  when  adjusting  the  power  contrast 
to  a  value  that  keeps  the  error  ratio  below  10“ 3. 

Point  A  is  representative  of  measurements  on  an 
experimental  model  with  correlation  detection  which 
had  only  10  character's  (the  10  figures).  This  reduced 
the  transmission  rate,  and  thus  the  bit  density,  to 
0.37  or  —4.4  db.  Considering  the  circuit  deficiency 
(filter  slopes)  brings  a  small  increase  in  the  transmission 
bandwidth,  giving  a  final  valuo  of  —5.0  db  for  the 
bit  density  of  the  experimental  model.  It  is  interesting 
to  note  that  points  A  and  B  are  almost  on  the  samo 
line  of  constant  power  contrast  ( +5  db).  This  examplo 
shows  how  the  utility  chart  can  bo  used  to  comparo 
experimental  designs  with  theoretical  models.  It 
likowiso  demonstrates  the  value  of  the  chart  when 
comparing  systems  with  different  transmission  rates. 

(2)  Operating  points  DEFOHI  :  This  area  is  representative 

for  laboratory  tests  of  U.S.  Army  data  transmission 


equipment.  Special  test  equipment  was  designed  for 
this  program  to  generate  data,  detect  errors,  and  record 
the  error  statistics  in  a  useful  manner  (Tucker  and 
Duffy  63).  Various  modems  (data  transmitter  and 
receivers)  were  tested  while  operating  over  carrier 
transmission  equipment  with  white  noise  as  disturbance. 

Points  DEF  are  for  binary  FSK  transmission.  They 
are  included  as  a  comparison  and  also  to  show  the 
influence  of  distortions  even  for  such  simple  waveforms. 
The  carrier  equipment  was  an  AN/TCC-7  12-channel 
set  for  field  cables.  Point  D  represents  the  results 
after  passing  one  channel  ;  point  E  is  reached  after 
passing  through  3  channels  in  fandom  ;  and  point  F, 
after  passing  12  channels  in  tandem. 

Points  GH1  are  the  results  when  sending  the  samo 
messages  over  a  four-phase  modem  at  1200  bits  per 
second. 

(3)  Operating  points  JKL  are  examples  of  operational 
results  to  be  discussed  as  a  demonstration  of  the  ad¬ 
vantages  of  the  utility  chart.  The  data  is  extracted 
from  a  recent  tabulation  of  the  operational  communica¬ 
tions  results  of  the  U.S.  deep  space  flights  of  Pioneer 
and  Mariner  (Roiff  66).  The  derivation  of  the  bit 
density  and  the  utility  from  this  publication  presents 
some  difficulty  as  the  paper  does  not  give  full  informa¬ 
tion  about  the  bandwidth  of  the  data  channels. 
The  difficulties  in  deriving  the  utility  result  because  a 
part  of  the  transmitter  power  goes  into  the  synchroniza¬ 
tion  channel  and  the  ranging  channel.  A  term  called 
“  modulation  loss  ”  gives  the  signal  power  in  db  which 
has  to  be  deducted  for  these  purposes.  Considering 
this  loss  and  using  the  signal  and  noise  levels  specified 
in  the  paper,  one  arrives  at  a  utility  of  —11  db  for 
Pioneer  V  and  Pioneer  VT  (point  .T),  of  —12  db  for 
Mariner  II  (point  K),  and  —13  db  for  Mariner  IV 
(point  L). 

This  examplo  may  show  that  it  is  possible  to  place 
operating  points  for  rather  complex  systems  on  the 
utility  chart.  In  this  case  we  leant  that  the  space 
data  systems  are  the  systems  with  the  lowest  bit 
density  of  all  the  practical  systems  that  are  considered 
in  this  set  of  examples.  This  was  to  be  expected, 
because  such  systems  operate  with  very  small  power 
contrast  (SNR).  Evidently,  this  is  the  top  requirement 
for  space-to -earth  links.  Indeed,  the  point  J  has  the 
lowest  power  contrast  (+3  db)  of  all  examples  in  this 
sot.  On  the  other  hand,  the  Pioneer  /Mariner  examples 
may  servo  as  an  introduction  to  the  complicated 
problems  of  deriving  bit  density  and  utility  from 
practical  measurements  on  operational  systems.  A 
more  detailed  analysis  of  the  actual  equipment  would 
possibly  place  these  systems  more  to  the  left. 

(4)  Operating  lines  MN  :  Line  M  evidences  the  perfor¬ 
mance  of  the  most  frequently  applied  nonbinary 
modulation  method — the  four-phase  digital  data  trans¬ 
mission,  also  called  the  quaternary  phase  keying  mode. 
In  particular,  the  lines  aro  derived  from  a  test  report 
of  the  Hughes  Aircraft  Company  (HC  270)  modem, 
a  digital  data  receiver  (Evans  et  al  61).  Lino  N 
indicates  results  of  a  vestigial  sideband  (VSB)  binury 
modem  as  a  comparison.  Linos  M  and  N  apply  to 
field  tests  over  leased  telephone  lines  (length  estimated 
by  this  author  at  250  miles).  The  lengths  of  the  lines  M 
and  N  indicate  the  margin  which  should  be  reserved 
when  different  kinds  of  leased  lines  would  bo  used. 
The  VSB  system  (line  N)  required  carefully  equalized 
line  facilities  while  the  nonbinary  system  (line  M) 
was  able  to  oporate  over  nonequalized  facilities.  This 
is  the  reason  that  the  margin  for  N  is  smaller  :  anticipat¬ 
ing  that  the  equalization  procedure  will  make  different 
kinds  of  lines  nearly  equal.  The  VSB  system  (N) 
needed  2160  Hz  bandwidth  to  operate  at  2400  bps 
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while  the  4-phaso  system  (M)  needed  only  1800  Hz 
to  operate  at  2500  bps.  This  is  why  line  M  is  placed 
approximately  1  db  to  the  right  of  line  N. 

This  example  clearly  demonstrates  the  advantages 
of  a  nonbinary  system — higher  hit  density  and  higher 
utility.  More  will  be  said  in  section  H  about  MI’SK 
systems. 

(5)  Operating  lines  OP  :  These  are  results  from  tests 
performed  in  Germany,  also  with  4-phaso  data  trans¬ 
mission  equipment,  hut  with  pulsive  disturbances 
intentionally  inserted  (Von  Hanisch  and  Ivettel  03). 
The  line  O  is  for  a  coherent  detection  system  ;  line  1’ 
is  for  a  noncoherent  detection  method.  The  upper 
ends  of  both  lines  are  the  points  for  bit  error  ratio 
of  Id-*  ;  the  lower  ends  are  the  points  for  a  hit  error 
ratio!  of  10- 6. 

These  German  tests  indicate  the  rather  disturbing 
influence  of  pulsive  noLse  on  data  transmission  systems, 
particularly  when  a  very  low  error  ratio  is  desired. 
The  (utility  is  brought  down  to  —2(5  db. 

(6)  Operating  line  QR :  This  last  example  has  been 
selected  as  an  interesting  case  introducing  a  novel 
fading  model  and  demonstrating  the  theoretical 
performance  of  certain  nonbinary  systems  over  channels 
following  the  characteristics  of  this  model.  The 
model  is  called  the  BFC,  the  binary  fading  channel 
(Motley  and  Melvin  60). 

This  example  is  intended  to  show  that  we  can  com¬ 
pare,  in  the  same  utility  chart,  the  performance  of 
systems  operating  over  linear  channels  and  others 
operating  over  nonlinear  channels,  provided  that  a 
mathematical  model  is  defined  for  tho  nonlinear 
channel  (BFC,  in  this  case). 

In  summarizing  the  common  aspects  of  all  the  six  examples, 
it  can  be  I  stressed  that  the  utility  chart  emerges  not  only 
us  a  practical  tool  for  the  comparison  of  binary  and  non- 
binary  modulation  methods,  hut  also  for  the  comparison 
between  any  digital  systems  with  different  bandwidths, 
different  rates,  or  different  coding  procedures.  .SNR, 
bandwidth  requirement,  transmission  rate,  and  error  ratio 
can  he  readily  read  off  from  the  utility  chart.  More  about 
the  utility  chart  and  its  applications  may  he  found  in  Fili- 
powsky  67c,  67d,  G8d. 

C3.  Threshold  Systems 

The  last  of  the  six  characteristics  which  are  used  to  describe 
nonlinear  data  systems  was  introduced  ns  “  distortion 
measures]”.  This  was  supported  by  a  statement  that,  for 
the  purpose  of  this  paper,  linear  operation  of  the  channels 
will  be  assumed  and  distortions  measures  will  he  introduced 
as  a  second-order  correction.  We  now  amplify  this  statement 
to  include  two  purposely  introduced,  severe  nonlinearities 
which  do  not  violate  the  assumption  of  a  basically  linear 
channel.  They  will  be  summarized  under  this  heading  of 
threshold:  systems. 

The  first  nonlinearity  is  the  threshold  in  the  decision  system. 
At  t  he  cud  of  t  he  linear  transmission  channel,  there  must  be  a 
device  that  restores  the  original  digital  character  of  the 
messages!  This  device,  usually  called  the  decision  system, 
is  characterized  by  the  error  probability  function,  which  will 
he  discussed  later.  Sometimes  decision  systems  are  used 
in  the  t  ansmitfer  for  converting  analog  information  into 
digital  form.  This  A/1)  conversion  process  causes  the  so- 
called  quantization  noise  (Crow  62).  In  this  paper  wo  are 
not  concerned  with  quantization  noise,  nor  with  the  closely 
related  distortions  and  noise  due  to  the  sampling  process 
(Stewart  56).  In  all  eases  we  assume  digital  input  and  digital 
output  for  the  nonbinary  systems. 

The  second  purposely  introduced  nonlinearity  is  the 
improvement  threshold  in  PM  systems.  Many  nonbinary 
systems  (will  operate  over  basically  linear  channels  which  go 


over  receivers  with  completely  nonlinear  IF  amplifiers 
(containing  chains  of  limiters).  Still  we  have  to  consider 
these  facilities  as  linear  channels  as  far  as  the  information¬ 
carrying  signals  are  concerned.  However,  these  channels 
must  not  be  treated  as  Gaussian  channels  as  far  as  the  noise 
is  concerned.  The  limiters  change  the  statist  ical  characteris¬ 
tics  of  the  noise.  The  utility  chart  can  he  used  in  such  cases 
by  combining  it  with  the  signal  processing  characteristic 
of  the  nonlinear  receiving  system  (Filipowsky  and  Muehldorf 
65a,  Section  4.1.4).  A  recent  paper  (Klapper  66)  provides  a 
link  between  the  threshold  mechanism  in  analog  FM  systems 
and  tho  error  ratios  in  digital  FM  reception.  An  earlier 
paper  (Von  Baeyer  63)  considered  the  same  problem,  in¬ 
cluding  the  effect  of  intersymbo!  distortions  due  to  band 
limitations.  Two  other  investigations  may  he  mentioned 
as  examples  of  the  many  studies  that  have  been  devoted 
to  FM  threshold  problems  (62,  Campbell  and  Herbert 
65).  Finally,  it  should  be  pointed  out  that  phase  lock 
loop  circuits  show  also  a  typical  improvement  threshold, 
whose  influence  on  digital  transmission  signals  may  be  of 
importance  (Becker  et  al  65).  Beyond  these  few  references 
we  must  omit  the  very  complex  nonlinear  problem  from 
this  paper  except  for  the  first  nonlinear  problem,  the  decision 
threshold. 

To  analyze  the  decision  process,  one  can  use  the  conven¬ 
tional  error  curves  which  are  plotted  in  most  publications 
and  which  are  derived  for  a  special  threshold  decider  model. 
For  example,  fig.  2  shows  on  the  right  side  the  error  curve 
for  an  ideal  binary  system,  marked  n  =  1 .  This  chart 
shows  the  utility  vertically  in  decibels  u  =  10  log,„  (N0/E) 
and  the  bit-error  ratio  es  horizontally.  The  other  curves 
on  t  he  right  side  (n  =  2,  3,  4,  6,  8,  1 0)  are  tho  error  probability 
curves  for  the  most  efficient  nonbinary  system,  Viterbi’s 
coded  phase-coherent  communications  system  (Viterbi  61). 
The  left  side  of  fig.  2  is  a  ut  ility  chart  as  discussed  in  section  C2. 
Tho  lines  A-E  are  the  utility  curves  for  an  ideal  binary 
system  in  DSB-SC  (double  sideband,  suppressed  carrier 
modulation).  It  is  assumed  that  coherent  detection  can 
take  place  and  that  t  he  signal  extractors  will  be  matched 
filters  matched  to  two  orthogonal  waveforms.  We  also 
assume  a  zero -threshold  decision  device,  which  will  have  the 
error  curve  marked  n  =  1  at  the  right  side.  Curve  A  is 
then  the  utility  curve  for  a  bit-error  ratio  of  10_1  ;  curve  B, 
for  1 0 - a ;  and  so  on  until  curve  E,  which  is  plotted  for  a 
constant  hit-error  ratio  of  10_s.  One  can  now  read  off 
how  many  db  utility  one  loses  if  the  error  ratio  should  he 
improved,  say,  from  10" 3  to  10“ 4  (only  about  1  db),  while 
an  improvement  from  10-*  to  lO-2  would  cause  a  utility 
loss  of  4.7  db.  To  find  these  curves  when  the  error  curve 
is  given,  one  needs  merely  search  for  the  point  on  tho  error 
curve  which  corresponds  to  the  desired  error  ratio.  This 
is  the  constant  utility  value  for  the  respective  utility  curve. 
The  particular  nature  of  the  ideal  binary  system  assures  the 
horizontal  run  of  these  curves. 

To  demonstrate  how  more  general  utility  curves  are 
derived  from  error  curves,  we  use  Viterbi’s  error  curves. 
From  Viterbi’s  paper  one  can  derive  the  following  relation¬ 
ship  : 


J)/< 


Kb 

B 


n 


0,1-1 


(24) 


D  f;  -  10  logl0  a  -  3.01  (n  —  1)  (24a) 


A  coded  phase-coherent  communications  system  of  n  —  6 
has  accordingly  M  —  64  different  orthogonul  sequences 
and  operates  with  a  hit  density  of  —7.27  db.  Tho  utility 
for  10- 3  error  ratio  can  be  guined  from  tho  error  curve 
marked  n  =  6.  It  is  —4  db.  This  would  give  one  point 
of  the  utility  curve  for  10-3.  However,  there  is  one  practical 
aspect  which  should  bo  considered.  Equations  24  have  been 
derived  by  Viterbi  under  the  assumption  that  the  system 
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operates  with  minimum  bandwidth.  He  considered  only 
one-half  the  elementary  rate  as  the  transmission  bandwidth. 
This  assumption  is  justifiable  if  the  system  operates  in  a 
wide  hand  channel  with  no  stringent  bandlimitutions. 
However,  if  the  system  should  operate  over  a  bandlimited 
channel  or  if  the  transmitter  is  restrained  from  radiating 
any  frequency  components  outside  the  nominal  transmission 
bandwidth,  distort  ions  will  be  unavoidable.  Such  distortions 
may  indirectly  cause  additional  errors  and  the  system  may 
require  a  much  higher  power  (operate  with  lower  utility) 
to  obtain  the  specified  low  error  rate.  The  optimum  design 
will  then  be  the  one  which  avoids  distortions  by  increasing 
the  bandwidth  when  higher  quality  is  needed.  This  is 
shown  in  fig.  2  by  shifting  the  utility  curves  to  t  he  left  when 
going  to  lower  error  ratio.  Curves  F  to  J  show  this  family 
of  utility  curves.  Because  of  this  offset  for  a  comfortable 
bandwidth  margin,  it.  happens  that  all  points  for  a  constant 
length  of  sequences  (n  const.)  lie  on  a  slightly  curved 
vertical  line  (dashed  lines).  Notice  also  that  for  n  I  the 
points  of  the  utility  curves  of  a  coded  phase-coherent  system 
are  also  the  right  endpoint  of  the  ideal  binary  DSB-SC 
system.  For  n  2  (quaternary  Viterbi  system),  the  bit 
density  is  the  same  as  for  n  1,  again  a  clear  advantage 


all  equipment  developers  who  hope  for  a  strong  market  ot 
their  new  systems. 

The  six  characteristics  developed  in  section  Cl  and  used 
directly  or  indirectly  in  the  utility  chart  are  also  the  six 
most  important  magnitudes  to  be  measured. 

The  transmission  rate  is  easy  to  measure  in  systems  with 
constant  rate.  They  are  normally  clocked  and  the  clocking 
rate  is  slaved  to  a  highly  accurate  master  clock.  Operation 
is  impossible  unless  transmitter  and  receiver  are  synchronized. 
Thus  the  receiver  follows  with  the  same  precision  that  the 
transmitter  has  in  generating  messages.  A  simple  checking 
of  the  total  transmission  time  with  a  stop  watch  and  the 
automatic  counting  of  the  total  number  of  transmitted 
message  elements  will  establish  the  average  rate.  Systems 
with  variable  or  adaptive  rate  present  only  a  slightly  more 
complex  problem,  but  averaging  of  the  message  count  over 
shorter  intervals  (seconds)  in  a  running  registration  of  these 
counts  will  provide  an  “  instantaneous  ”  rate  sufficiently 
accurate. 

The  error  ratio  is  moro  difficult  to  observe.  The  usual 
practice  is  to  loop  the  test  link  back  to  its  origin,  if  this  is  a 
feasible  approach  (Bart clink  and  Knight  04).  Transmitter 
and  receiver  can  thon  be  located  next  to  each  other,  and  input 


Piyurv  ticncrutiny  utility  curves  (left)  from  the  error  probability 
curves  ( riyltt ). 


for  the  nonbinary  system.  Notice  also  that  the  utility  is 
truly  improving  for  all  curves  with  increasing  length  of 
sequences.  However,  the  longer  the  sequences,  the  smaller 
is  the  improvement  for  another  doubling  of  the  length  of 
sequences.  An  optimum  length  will  soon  he  reached  when 
hardware  costs  have  to  bo  considered.  More  will  be  said 
about  these  systems  in  section  K. 

Decision  devices  may  also  have  the  capability  to  refuse  a 
decision.  This  is  normally  done  by  giving  the  device  two 
thresholds,  declaring  the  zone  between  the  two  thresholds  a 
null  zone.  If  the  observed  magnitude  at  a  decision  instant 
lias  a  value  in  the  null  zone,  no  decision  will  be  made  (Bloom 
et  al  57). 

C4.  Measurement  of  Performance 

Ultimately,  it  is  the  measurement  of  the  performance 
of  a  system  during  operation  that  will  decide  its  fate.  Binary 
systems  are  well  established  :  they  are  simple  and  their 
performance  is  good.  If  nonbinary  systems  do  not  demon¬ 
strate  equal  or  superior  performance,  any  cost,  speed,  or 
other  advantage  will  not  be  adequate  to  bring  the  sweeping 
success  that  some  people  predict.  The  understanding  of 
measurement  and  test  procedures  is,  therefore,  important  for 


messages  and  output  messages  can  be  directly  compared  in 
coincidence  circuits,  provided  the  former  ones  will  be  delayed 
for  an  amount  equal  to  the  total  loop  transmission  delay. 
This  procedure  also  gives,  as  a  by-product,  this  total  delay 
time.  Noncoincidence  of  message  elements  will  be  counted 
as  errors.  Precaution  must  be  taken  that  no  systematic 
noncoincidence  error  might  be  established  within  the  test 
equipment. 

Interesting  test,  facilities  have  been  developed  for  the 
statistical  recording  and  evaluation  of  errors.  Bason  ((ill) 
describes  equipment  for  the  experimental  evaluation  of  a 
system  with  multilevel  pulses  (MASK)  and  for  the  counting 
of  its  errors.  The  Hughes  Aircraft  Company  designed  a 
special  digital  error  measuring  set  (DBMS)  (60a)  and  Entick- 
nap  (61)  discusses  the  equipment  used  by  t  he  Lincoln  Labora¬ 
tory  of  MIT  for  data  transmission  tests  over  telephone 
circuits.  The  equipment  is  called  ADDER,  acronym  for 
Automatic  Digital  Data  Error  Recorder  (Hofmann  61). 
Maholick  (65)  reports  on  project  COMFACTS,  which  involved 
a  specially  designed  central  processor  similar  to  the  IBM  1401. 
The  processor  provides  specifications  to  describe  the  perfor¬ 
mance  and  reliability  of  any  data  transmission  installation 
under  test.  Small  (65)  applied  a  technique  that  provides 
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time  compression  in  the  recording  of  error  patterns.  This 
permits  longer  automatic  recording  periods  as  well  as  reduced 
testing  and  analysis  costs. 

The  cjjcrtive  transmission  bandwidth  is  in  many  cases  a 
standard  value  prescribed  by  the  public  carrier  or  any 
national  or  international  communications  administration. 
In  other  cases  it  must  be  derived  from  frequency  response 
j  measurements  with  the  help  of  a  simple  graphical  procedure. 

J  If  its  measurement  is  needed,  the  methods  are  the  same  as 
for  any  other  frequency  response  measurement. 

The  measurement  of  the  signal  power  would  be  a  simple 
procedure  with  standard  instruments  except  for  the  fact 
that  it  must  be  determined  at  a  rather  low  power  level, 
where  usually  noise  is  present.  Thus  one  actually  measures 
the  power  of  the  mixture  of  signal  and  noise.  If  it  is  also 
essential  to  know  the  power  distribution  (i.e.,  the  power 
spectrum  of  the  signals),  one  will  normally  measure  the  total 
spectrum  of  typical  messages  with  a  signal  analyzer  and 
determine  from  this  curve  the  signal  power  within  the  trans¬ 
mission  band.  Similarly,  it  is  possible  to  find  the  peak 
power  from  oscillographic  observations  of  the  time  functions 
of  typical  message  signals  or  from  direct  measurements  with 
the  PAR  (peak  to  average  power)  meter  (Anderson  (14). 
Attenuators  in  laboratory  tests  or  attenuation  measurements 
in  field  tests  must  then  establish  the  power  level  at  various 
places  in  the  system. 

The  noise  power  can  be  evaluated  when  the  signal  power 
is  temporarily  switched  off  the  system,  provided  the  channel 
is  perfectly  linear.  If  automatic  volume  controls,  limiters, 
and  other  nonlinearities  are  involved,  one  may  err  greatly 
when  concluding  that  the  noise  power  without  the  signal 
present  would  be  the  same  as  the  noise  power  during  the 
correct  operation  of  the  system. 

The  same  difficulties  apply  to  the  direct  measurement  of 
the  SNR  (Levitan  and  Peysikhman  64).  The  SNR  monitor¬ 
ing  of  deep  space  facilities  presents  particular  difficulties 
because  of  the  extremely  weak  signals  reaching  the  antenna. 
The  Deep  Space  Instrumentation  Facility  in  the  U.S.A.  has 
analyzed  two  automatic  monitoring  methods  of  high  precision 
(Gilchriest  64)  which  may  be  of  interest  for  application  in 
other  areas. 

Signal  I  energy  or  energy  contrast  are  rarely  measured 
directly.  The  duration  of  signals  or  messages  is  normally 
well  known  and  the  energy  can  be  calculated  when  the  average 
signal  powor  is  known. 

Particular  problems  are  created  by  pulsive  noise  (Bodonyi 
61  ;  Fenniok  65)  and  by  time-varying  channels  (Bukhviner 
64  ;  Sharkey  et  al  65  ;  Korte  65  ;  65a).  In  the  first  case, 
one  usually  resorts  to  laboratory  tests  with  pulsive  noise 
recorded  on  magnetic  tapes  as  it  has  been  observed  over 
the  circuits  to  be  used  (Fennick  65).  In  the  second  case, 
special  measurement  and  recording  methods  are  applied 
to  explore  the  statistical  behaviour  of  the  medium.  Once 
this  is  known,  simulation  methods  are  applied  in  the  labora¬ 
tory  to  duplicate  this  behaviour  under  well-controlled 
conditions.  In  both  cases,  the  experimenter  should  operate 
the  different  test  objects  (competitive  system)  under  realistic 
but  practically  identical  test  conditions. 

Simulation  is  also  applied  to  compare  various  error- 
correcting  methods  under  realistic  pulsive  noise  conditions 
(Bennett  and  Froehlich  61).  Eye  patterns  are  a  relatively 
new  method  of  a  more  subjective  inspection  of  transmission 
channels  and  modulation  methods  with  the  help  of  oscillo¬ 
graphic  devices  (Davey  64,  Von  Hiinisch  and  Kettol  63). 

Complete  test  installations  for  the  evaluation  of  data 
systems  over  telephone  and  telegraph  circuits  are  described 
by  Purtov  et  al  (65)  for  testing  Russian  facilities,  by  Ebenau 
and  Stotesbury  (64)  for  testing  West  European  facilities, 
by  Tucker  and  Duffy  (63)  and  Bartow  (65)  for  testing  U.S.A. 
line  modems  for  Army  applications,  and  by  Lamb  (63)  for 


testing  U.S.A.  Army  mobile  tactical  equipment.  A  Russian 
paper  reports  a  method  for  a  statistical  study  of  telegraph 
characteristics  using  a  digital  computer  (Tyakhti  65). 
Ansorgo  (65)  describes  a  general  test  sot  for  the  evaluation 
of  the  quality  of  a  voice-frequency  data  transmission  set 
and  staff  members  of  the  Jet  Propulsion  Laboratory  (JPL) 
investigated  the  characteristics  of  test  messagos  for  super¬ 
vising  their  world-wide  surface  communications  network  on 
Earth  (65b). 

The  above-mentioned  publications  are  only  a  small  fraction 
of  all  the  reports,  papers,  and  descriptions  of  tests  and  test 
facilities  for  communications  plants.  Those  selected  are 
either  directly  concerned  with  testing  of  nonbinary  systems 
or  they  develop  the  material  which  we  think  is  of  importance 
for  this  purpose. 


D.  Mathematical  Models 

Mathematical  models  are  indispensable  for  the  analysis  of 
any  electronic  system.  For  nonbinary  systems  they  actually 
arc  the  point  of  origin.  Many  new  ideas  have  evolved 
from  mathematical  models,  and  many  more  papers  have  been 
written  about  the  mathematical  models  than  about  the 
practical  application  of  nonbinary  systems. 

A  mathematical  model  can  best  be  understood  when  ex¬ 
plained  in  terms  of  a  block  diagram.  Figure  3  is  an  attempt 
to  show  the  functional  units  of  a  very  general  nonbinary 
system  operating  over  a  single  channel.  Let  us  explain  this 
diagram  in  connection  with  an  octonary  MFSK  system. 
This  is  a  system  where  eight  different  frequencies  are  available 
to  carry  the  information. 

Figure  3a  shows  the  five  key  elements  in  any  nonbinary 
system. 

Figure  4  shows  the  sequences  of  operation  in  the  particular 
MFSK  system.  The  encode  block  separates  nine  binary 
input  elements  into  three  trigrams  A,  B,  C  and  forwards 
these  to  the  signal  formation  block.  Following  the  signalling 
alphabet  on  the  top  of  fig.  4,  the  signal  formation  device 
will  create  frequencies  f3 ,  f, ,  and  f6 ,  which  go  in  this 
sequence  over  the  channel  where  the  signals  are  contaminated. 
The  extraction  device  on  the  receiving  side  will  give  an 
output  signal  for  each  of  the  possible  eight  frequencies. 
After  the  first  signal  A"  arrives,  assume  that  there  is  only 
one  output  voltage  (of  peak  value)  at  the  output  for  f3  . 
All  other  outputs  are  zero.  This  corresponds  to  a  noiseless 
and  distortion-free  transmission.  Afer  the  second  signal 
B"  arrives,  there  are  several  output  voltages,  but  the 
highest  is  still  the  one  of  frequency  fi,  which  was  actually 
transmitted.  After  the  third  signal  C"  arrives,  there  are 
two  outputs  higher  than  the  others — f2  (highest)  and  fB 
(second).  As  we  know  that  f6  has  been  transmitted,  we 
may  anticipate  that  the  receiver  will  make  an  error. 

Indeed,  if  the  following  decision  and  decoding  device 
follows  the  so-called  maximum  likelihood  rule,  it  will,  for 
each  decision,  compare  all  the  eight  outputs,  determine  the 
largest  one,  and  conclude  that  this  is  the  signal  that  has 
actually  been  transmitted.  However,  the  information 
might  have  been  encoded  in  more  complex  form,  permitting 
the  decider  to  perform  more  complex  calculations  which, 
under  certain  circumstances,  can  tolerate  large  errors  or 
erasures  in  some  of  the  signals  of  a  largo  group  and  still 
arrive  at  a  correct  final  decision. 

In  the  foregoing  example,  wo  assume  that  the  decider  is 
bound  to  make  an  error  when  interpreting  the  last  digit 
and  will  deliver  to  the  output  the  group  +  H —  (f2)  in  placo 
of  the  correct  group  — | —  (f6).  Notico  that  an  error  in 
the  interpretation  of  an  octonary  digit  may  cause  either  one, 
two,  or  three  binary  output  errors.  If  the  equipment  also 
considers  the  runners-up  (i.e.,  the  outputs  with  the  second 
and  third  highest  output),  it  frequently  can  assure  that  single 
binary  errors  result  from  a  transmission  error  and  that 
multiple  errors  are  avoided.  This  is  why  it  Is  impossible  to 
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Figure  3. — Block  diagram  of  a  general  nonbinary  transmission  link. 
ENCODING  PROCESS  DECISION  PROCESS 


Largest  Output  at 


EBBEBEBE 

□DDBDDHE 

DBBBDBDE 

|Jj|BB0|300Q 


1. 

Input. 

2. 

3. 

Transmit 

EXAMPLE 


Figure  4. — An  example  of  the  nonbinary  transmission  of  a  9-bit 
message  over  an  octonary  MFSK  system. 


establish  a  fixed  relationship  between  transmission  character 
error  ratio  and  output  bit  error  ratio  without  knowing  the 
decoding  procedure.  Upper  and  lower  limits  for  this  relation¬ 
ship  are  quoted  in  equation  2. 

Assuming  that  the  eight  transmission  signals  are  orthogonal 
to  each  other,  the  approximate  power  contrast  can  be  deter¬ 
mined  by  dividing  the  square  of  the  highest  output  by  tho 
sum  of  the  squares  of  all  lower  outputs.  The  results  are 
marked  as  a  db  in  fig.  4. 


Returning  to  fig.  3,  part  b  (transmitter)  and  part  c  (receiver), 
one  sees  the  essential  functional  elements  within  each  of  the 
five  units  of  fig.  3a.  The  encoding  unit  consists  of  a  serial 
to  parallel  converter  (S/P),  the  actual  encoder,  a  block  of 
hold  circuits,  and  the  master  clock.  The  latter  can  be 
slaved  to  the  input  rate  via  line  2  ;  it  will,  in  turn,  control 
the  rates  of  all  operations  of  the  encoding  subsystem.  The 
clock  drives  the  signal  generator  via  line  1 1  and  secures 
the  transmission  of  synchronization  information  over  tho 
channel  to  the  receiver.  This  is  symbolized  by  connection  8. 
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Tho  signal  formation  subsystem  consists  basically  of  the 
signal  generator  and  tho  modulating  matrix.  Examples 
will  be  given  in  the  following  sections  of  the  design  of  these 
units  for  i  ho  various  nonbinary  systems.  In  the  example 
of  tho  octonary  MFSK,  the  signal  generator  produces  the 
eight  frequencies  at  parallel  outputs,  feeding  them  over 
eight,  connections  (12)  to  the  modulation  matrix.  This 
device  consists  merely  of  eight  switches,  each  of  which  passes 
the  corresponding  frequency  to  the  transmitter  (Xmtr) 
when  triggered  from  the  hold  block.  Connection  10  likewise 
consists  cjf  eight  linos.  1  f  the  output  bandwidth  is  to  be 
strictly  limited,  a  shaping  pulse  may  bo  sent  over  line  13 
to  suppress  the  output  amplitude  smoothly  at  tho  time  of 
the  sudden  transition  of  one  frequency  to  the  other. 

The  transmission  channel  is  symbolized  by  a  block  repre¬ 
senting  signal  delay  and  signal  distortions,  a  linear  adder 
superposing  the  disturbances  (noise)  and  the  signal,  and, 
finally,  another  block  representing  the  time  variations 
(multiplicative  disturbances). 

The  receiving  terminal  consists  of  an  extraction  section 
and  a  decision  section.  The  former,  as  the  name  implies, 
extracts  the  class  of  desired  signals  from  any  other  signals 
which  may  reach  the  receiver  input.  This  is  normally  done 
by  selective  amplifier  circuits,  which  pass  only  the  desired 
transmission  band.  Low  noise  performance  of  this  most 
sensitive  part  of  a  link  is  essential.  In  particularly  sensitive 
installations,  there  are  special  circuits  for  the  extraction 
of  the  Transmission  carrier.  In  extreme  cases,  one  must 
go  through  special  carrier  acquisition  procedures  (Iti)  before 
the  carrier  extractor  can  be  activated.  The  carrier,  once 
restored!  in  tho  receiver,  can  be  used  to  improve  the  signal 
extraction  process.  This  is  indicated  by  connection  19, 
which  leads  to  the  signal  processor.  This  term  is  used  for 
widely  differing  devices.  We  restrict  its  meaning  to  in¬ 
corporate  all  nonlinear  analog  operations  on  the  transmission 
signals.  These  are  particularly  important  whenever  tho 
information  is  carried  in  angular  modulation.  They  are 
normally  known  as  limiters,  discriminators,  signal  or  fre¬ 
quency  tracking  devices,  etc.  (See  section  C3.)  Sometimes 
it  is  difficult  to  specify  exactly  which  circuits  of  a  receiving 
installation  should  bo  included  into  a  particular  one  of  the 
three  units  :  carrier-hand  receiver,  signal  processor,  and 
carrier  [extractor. 

The  demodulating  matrix  is  usually,  hut  not  always,  the 
counterpart  to  the  modulating  matrix.  Most,  systems 
operate  with  the  one  or  the  other  kind  of  a  synchronous 
(coherent)  demodulator.  For  this  operation,  noise-free  and 
distort  ion-free  replicas  of  the  transmitted  signals  are  generated 
locally  in  the  receiver.  They  are  then  multiplied  with  the 
noisy  received  signals  and,  in  many  cases,  the  products 
are  integrated  and  stored  in  the  “  hold  ”  circuits.  To  perform 
this  operation  correctly,  the  signal  generator  must  be  phased 
exactly  to  the  incoming  signals.  This  is  done  via  connection 
22  ;  the  operation  is  normally  called  clocking  acquisition  or 
epoch  acquisition  or  simply  synchronization.  Once  the 
demodulator  matrix  is  in  perfect  operation,  the  signal 
generator  can  be  locked  in  by  a  closed-loop  control  operation 
to  maintain  perfect  clocking  even  under  time  variations 
(Doppler)  of  the  transmission  channels  (24). 

The  decision  part  of  the  receiver  consists  of  the  decider/ 
decoder  unit,  a  hold  circuit  block,  and  a  parallel  to  serial 
converter  (P/S)  unit.  All  those  units  must  be  slaved  to 
the  receiver  clock  via  connections  27  and  28. 

The  receiver  operation  in  the  octonary  MFSK  system 
can  be  simplified.  In  the  extreme  case  one  would  retain 
only  [the  demodulating  matrix,  the  decider,  and  the  output 
circuitry.  The  former  would  consist  of  eight  bandpass 
filters,  each  tuned  to  one  of  tho  eight  frequencies.  Each 
of  t  hese  would  be  followed  by  a  full-wave  envelope  detector 
with  i  a  time  constant  approximately  equal  to  the  duration 


of  one  elementary  octonary  signal.  The  decider  could  be 
self-synchronized  to  the  peaks  of  the  demodulator  output. 
Beyond  this  oversimplified  receiver  arrangement  there  are 
many  more  complex,  but  also  more  efficient,  versions, 
using  all  the  other  special  circuits  displayed  in  fig.  3. 

With  this  general  block  diagram  of  a  nonbinary  trans¬ 
mission  system  (fig.  3),  one  can  now  look  for  mathematical 
models  describing  the  key  units.  Notice  that  several  magni¬ 
tudes  in  these  models  will  be  random  processes.  There  are 
many  good  textbooks  available  on  statistical  mathematics 
and  on  t  he  fundamentals  of  random  processes.  Particularly 
suitable  for  the  requirements  of  communications  systems 
are  a  recent  Russian  review  (Tikhonov  64),  a  German  review 
(Von  Szalay  94),  and  an  unclassified  U.S.  Government 
report  (Goodman  (14). 

The  most  general  mathematical  model  of  a  nonbinary 
system  results  from  C.  E.  Shannon’s  historically  famous 
paper  (Shannon  49).  It  derives  the  Tipper  bound  of  the  bit 
density  of  an  ideal  not  further  specified  communications 
system  with  the  help  of  multidimensional  geometrical  map¬ 
ping  considerations.  Many  of  the  more  specific  mathematical 
models  to  be  discussed  in  the  next  sections  go  back  to  this 
fundamental  paper.  A  more  recent  paper  should  be  of 
highest  interest  to  analytical  workers  as  it  attempts  the 
partial  ordering  of  nonbinary  channels  (Chang,  T.  T.,  and 
Lawton  62).  This  paper  contains  further  references  to  other 
fundamental  papers  on  this  critical  problem  which  have 
been  published  by  C.  E.  Shannon  and  other  in  the  14  years 
between  these  two  publications.  J.  K.  Wolf  (62)  points 
out  that  a  clarification  of  terminology  and  an  agreement  on 
several  fundamental  magnitudes  in  noise  theory  is  essential 
when  comparing  mathematical  models  of  different  authors. 

Floyd  and  Nut  tall  (65)  published  in  May  1965  a  U.S. 
Government  report  which  is  available  to  the  public.  They 
prove  that  nonbinary  systems  are  always  superior  to  binary 
systems  even  if  the  latter  ones  use  error-correcting  coding 
procedures.  In  this  report  is  a  rather  elaborate  mathematical 
model  which  applios  four  different  levels  of  nonbinary 
coding  procedures. 

All  the  investigations  mentioned  so  far  present  mathe¬ 
matical  models  of  the  most  general  nature.  They  do  not 
specify  special  waveforms,  nor  special  modulation  methods 
or  circuits.  This  will  be  done  in  the  remaining  Parts  of  this 
paper. 

The  staff  of  JPL  has  an  interesting  answer  for  those  who 
should  doubt  the  usefulness  of  mathematical  models  in 
general.  They  compared  Viierbi’s  model  of  an  orthogonal 
coded  phase-coherent  nonbinary  system  with  the  results  of 
precision  measurements  and  found  the  agreement  to  be 
very  satisfactory  (63a). 

1)1.  Transmitter  Models 

Tho  transmitter  model  specifies  the  nonbinary  system 
and  prescribes  coding  procedures  and  signal  waveforms. 
Modulat  ion  methods  are  usually  included  in  the  specification 
of  the  signal  waveforms.  When  specifying  the  transmitter 
model,  one  must  know  the  predictable  channel  characteristics, 
and  the  transmitter  model  should  match  the  channel  charac¬ 
teristics  in  the  most  efficient  way.  Usually  there  are  several 
receiver  models  that  can  be  used  with  a  given  transmitter 
and  a  given  channel  model.  One  of  these  will  be  optimum 
in  a  theoretical  sense,  although  a  different  one  may  be  the 
most  efficient  in  an  economical  sense.  Utility  curves  will 
present  the  total  efficiency  of  any  selected  combination  of 
the  three  kinds  of  models. 

Shannon  (49)  hinted  at  a  transmitter  model  in  connection 
with  a  block  diagram,  similar  to  the  one  given  in  fig.  3, 
by  discriminating  bet  ween  the  message  space  and  the  signal 
space.  The  coding  subsystem  of  fig.  3  can  be  taken  as  the 
device  forming  the  most  suitable  message  space  from  the 
original  information  flow.  The  signal  formation  subsystem 
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can  be  taken  as  the  device  forming  Shannon’s  signal  space. 
For  the  latter  one  Shannon  states  that  “  It  can  be  shown 
that  (its)  coordinates  are  all  perpendicular  to  each  other 
and  are  obtained  by  what  is  essentially  a  rotation  of  the 
original  coordinate  systems.”  As  his  original  coordinate 
system.  Shannon  used  the  system  of  sin  x/x  funct  ions  equally 
spaced  r  =  1/2B  seconds  apart.  His  foresight  clearly 
indicates  that  the  mathematically  convenient  sin  x/x 
system  will  not  be  the  ideal  coordinate  system  for  practical 
cases  ;  rather,  he  suggested  a  rotation  of  the  coordinate 
system  to  arrive  at  a  more  practical  signal  space. 

What  Shannon  could  not  disclose  in  1 949  was  the  mechanism 
of  arriving  at  those  desirable  characterist  ics  that  could  bring 
a  practical  system  close  to  the  ideal  systems.  More  detailed 
mathematical  models  had  to  show-  the  path.  A  refinement 
w-as  needed  for  answering  a  question  in  connection  with 
Shannon’s  effect  number  5,  the  processing  delay  :  How- 
close  can  one  approach  the  ideal  system  when  tolerating  a 
specified  finite  delay  ?  Mathematically,  this  answer  required 
the  solution  of  the  “  sphere-packing  problem  ”,  which  was 
geometrically  introduced  by  Gilbert  (52)  in  his  search  for 
optimum  signal  alphabets  in  the  sense  of  Shannon’s  1949 
recommendations.  The  sphere-packing  problem  received 
more  complete  attention  by  Balakrishnan  (61)  and  Weber  (65). 
Technically,  the  easier  accessible  approach  to  optimum 
systems  with  finite  delay  seemed  to  be  the  use  of  the  more 
familiar  coding  techniques  with  binary  elements. 

Again  it  was  Shannon  (57)  who  attempted  in  mathemat  ical 
studies  to  arrive  at  a  model  of  an  ideal  system  under  the 
restraint  of  a  finite  coding  interval.  His  1959  paper,  en¬ 
hanced  by  Slepian’s  practical  interpretation  (Slepian  63), 
provides  such  a  model.  Figure  5  is  a  representation  of 
Slepian’s  charts  in  the  units  used  for  the  utility  plot  of  this 
paper.  Slepian  uses  six  magnitudes,  similar  to  the  six 
magnitudes  of  section  C 1  in  the  present  paper.  He  performed 
a  computer  evaluation  of  the  improvement  which  an  ideal 
nonbinary  system  could  reach  over  a  binary  system.  He 
assumed  a  gradually  increasing  encoding  delay  which  would 
not  reach  the  infinite  delay  postulated  by  Shannon  in  1949 
as  the  fifth  characteristic  of  the  ideal  system. 

Shannon  (59)  and  Slepian  63)  use  the  maximum  number 
of  dimensions  of  the  signal  space  as  specified  by  Shannon 
in  1949  and  earlier  by  Nyquist  in  1928,  i.e.,  the  so-called 
Nyquist  rate  2)3ir  =  2BTn-  .  They  define  “  block  codes  ” 
of  length  l0  equal  to  the  number  of  dimensions  of  a  signal. 
Section  lv  will  give  full  details  of  the  Shannon/Slepiau  results. 

We  see  in  fig.  5  that  the  limitation  of  the  encoding  interval 
to  five  dimensions  forces  a  dcsignor  to  be  satisfied  with  a 
utility  of  at  least  5  db  below-  the  ideal  system  when  bis 
requirement  is  to  have  less  errors  than  1  in  10,000.  For 
an  error  ratio  of  10-*,  he  pays  a  penalty  of  at  least  6.4  db 
below-  the  ideal  system.  It  is  also  interesting  to  know-  that 
the  smallest  penalties  apply  to  systems  with  3  db  and  5  db 
bit  density.  Systems  with  low-er  or  higher  bit  density 
require  a  still  higher  penalty.  One  can  approach  the  ideal 
system  closer  when  extending  the  length  of  the  block  codes 
to  the  10  =  25  dimensions  and  still  closer  for  1  =  101.  Those 
10  values  happen  to  bo  the  values  for  which  Slepian  (63)  has 
reached  a  numerical  solution  of  Shannon’s  (59)  equations. 

The  historial  efforts  to  follow  the  guidance  of  the  theory 
included  such  primitive  modes  as  passive  ternary  systems 
(Filipowsky  58)  and  proceeded  through  the  interesting  ideas 
of  case  coding  (Bason  64)  to  the  linear  real  coding  approach 
of  Pierce  (66)  and  the  permutation  modulat  ion  of  Slepian  (65a) 
to  be  discussed  in  Part  III. 

The  general  concept  of  signal  space  as  a  multidimensional 
space  has  received  much  attention,  not  only  for  the  purpose 
of  nonbinary  communications  systems.  A  few-  of  the  many 
papers  in  this  special  mathematical  discipline  may  bo  of 
particular  interest.  Ivirshner  (62)  described  the  interrelation- 


Fignre  5. — Utility  curves  of  an  Heal  nonbinary  system  with  5. 
25  or  101  dimensional  signals  and  error  ratios  of  10-4  awl  10"*. 


ship  between  signal  space,  modulation,  and  bandwidth  in  a 
rather  practical  manner  which  may  appeal  to  many  engineers. 
Bello  (64)  developed  a  concept  of  duality,  called  time-fre¬ 
quency  duality,  which  is  applicable  to  a  class  of  networks 
called  communieations-signal-proeessing  networks.  He  il¬ 
lustrated  the  benefit,  of  this  approach  by  constructing  the 
dual  of  the  Kineplex  communications  system.  (See  sec¬ 
tion  H.)  M.  Zakai  (65)  investigated  the  problem  of  band- 
limited  functions  and  bandlimited  (but  not  necessarily 
stationary)  processes  in  relation  to  the  sampling  theorem. 
There  are  also  interesting  recent  contributions  by  Kailath 
(65),  Huang  and  Gabrielian  (66),  and  R.  S.  Bennett  (66). 

Turning  now  from  the  mathematical  models  of  the  message 
encoding  susbvstem  (fig.  3)  to  those  of  the  signal  formation 
subsystem,  one  must  first  recognizo  the  basic  problems  of 
generating  one-dimensional  bandlimited  signals.  Most  systems 
arrange  these  signals  in  time  sequence,  which  raises  the 
problem  of  intersymbol  interference.  We  briefly  referred  to 
this  problem  in  section  Bl.  The  mathematical  model  of 
the  signal  format  ion  specifies  the  kind  of  signals  which  cause 
the  minimum  intersymbol  interference  and  still  permit  a 
high  signal  density.  Practical  signal  design  will  remain, 
to  some  degree,  an  art,  and  adjustments  w-ill  have  to  be  made 
to  match  the  signal  formation  subsystem  to  a  given  channel. 
Signal  space  diagrams  in  the  form  of  Lissajoux  figures  or 
eye  patterns  will  be  helpful  in  such  matching  efforts 
(Davoy  64). 

When  generating  mathematical  models  of  one-dimensional 
signals,  one  should  know  the  characteristics  of  so-called 
elementary  signals  and  of  their  signal  bases  (TB  products). 
Gabor  (46)  used  Gaussian-shaped  signals,  and  Ville  (48), 
Oswald  (56)  and  Silverman  (58)  investigated  analytical  signals. 
Those  are  mathematical  models  of  elementary  signals  which 
use  a  complex  time  domain  and  a  complex  frequency  domain. 
In  tills  way  it  is  possible  to  treat  pairs  of  related  signals 
(Hilbert  transforms)  in  one  and  the  same  analytical  operation. 
Tikhonov  anti  Lcvikov  (65)  compare  the  /3-values  (TB 
products)  of  a  number  of  pulsed  signals  including  rectangular 
signaLs,  Gaussian-shaped  signals,  and  transients  of  devices 
with  from  one  to  five  resonant  circuits.  Other  contributions 
to  the  analysis  of  the  shape  of  elementary  signals  are  available 
from  Chalk  (49),  Muehldorf  et  al  (59bj,  Gurovich  (60),  and 
Kennedy  (60).  Giovannoni  et  al  (65)  described  a  technique 
for  synthesizing  bandpass  waveforms  by  modulating  a 
sampling  waveform  at  several  instants,  i.e.,  actually  generat¬ 
ing  a  sequence  of  sampling  waveforms.  Weaver  (62) 
published  an  analysis  of  the  practical  aspects  of  signal 
formation  for  various  pulse  modulation  systems  and  con¬ 
cluded  that  a  signal  base  of  5  Is  required  when  intersymbol 
interference  shoidd  remain  below-  —30  db.  Levi  (65)  solved 
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the  problorn  of  fitting  an  arbitrarily  bandlimited  signal  to 
a  finite  number  of  arbitrary  points  in  the  amplitude-time 
domain  with  a  signal  of  minimum  energy.  An  interesting 
signal  with  dual  properties  is  described  by  Lender  (65a). 
These  signals  aro  elementary  signals,  which  also  can  be 
classified  as  multidimensional  signals.  Part  IV  will  give 
more  details  about  practical  transmission  signals. 

Mathematical  models  of  signal  formation  subsystems  for 
multidimensional  signals  are  more  essential  than  those  for 
one-dimensional  bandlimited  signals.  We  stressed  at  the 
beginning  of  this  section  that  a  system  approaching  tho  ideal 
system  requires  signals  of  very  high  dimensionality.  Tho 
designer  of  such  signals  now  has  full  control  over  the  inter¬ 
symbol  interference  between  the  components  of  the  signal. 
The  only  remaining  interference  out  of  his  control  is  the  one 
close  to  transition  points  between  adjacent  multidimensional 
signals.  Thus  we  see  that  the  importance  of  intersymbol 
interference  reduces  with  increasing  dimensionality  of  the 
signals.  The  practical  engineer  who  works  with  “  long  ” 
signals  is  not  much  concerned  with  their  “  fringe  effects  ”. 
On  the  oilier  hand,  signal  distortions  may  lead  to  misinter¬ 
pretations  of  some  of  the  signals  of  the  resulting  very'  large 
alphabets.  The  complexity  of  such  large  sets  of  signals 
makes  it  imperative  to  use  well-defined  mathematical  models. 

The  decreasing  influence  of  neighbouring  signals  permits 
the  application  of  a  ride  of  thumb,  linking  the  dimensionality 
of  a  signal  to  the  signal  base  that  Ls  required  to  transmit 
the  signal  with  reasonably  small  distortions  : 

p  —  TB  =  JLj  (25) 

1  Ls  the  dimensionality  of  the  signal  or,  in  other  words, 
the  number  of  orthogonal  coordinates  which  can  completely 
describe  a  signal  of  the  signal  base  p.  According  to  this 
rule,  a  one-dimensional  signal  (1  =  1)  requires  a  signal 
base  (TB  product)  of  1,  while  a  signal  of  large  dimensionality 
needs  only  a  signal  base  of  1/2,  i.e.,  of  half  the  dimensionality. 

Most  mathematical  models  of  multidimensional  signals 
follow  Shannon’s  example  and  describe  the  signals  as  vectors 
in  a  multidimensional  space.  Various  orthogonal  coordinate 
systems  may  be  used  and,  indeed,  one  arrives  at  an  orthogonal 
set  of  signals,  when  placing  all  signal  points  on  the  orthogonal 
axes  of  such  a  coordinate  system.  (Orthogonal  signal  sets 
are  discussed  in  greater  detail  in  Parts  III  and  IV.) 

Arthurs  and  Dym  (62)  explained  the  application  of  ortho¬ 
gonal  coordinates  with  the  help  of  an  example  in  two- 
dimensional  space.  Their  general  mathematical  model  is 
of  great  importance  for  the  theory  of  all  multistate  systems. 
Lachs  (63)  showed  how  to  maximize  the  minimum  distance 
for  a  set  of  m  points  on  an  n-dimensional  sphere,  a  problem 
which  has  to  bo  solved  when  designing  suborthogonal 
digital  systems. 

Many  investigators  concentrate  on  particular  orthogonal 
functions  as  the  basis  of  a  coordinate  system  for  multi¬ 
dimensional  signals.  Recently,  strong  attention  has  been 
given  to  systems  of  prolate  spheroidal  wavefunctions  as 
orthogonal  coordinate  systems.  (Details  will  be  discussed 
in  section  N4.)  Once  an  orthogonal  coordinate  system  is 
established,  interesting  rotational  transformations  can  be 
performed  on  the  complete  set  of  signals  to  match  the  signal 
characteristics  better  to  the  channel  characteristics  (Lang  63). 
Nuttull  and  Amoroso  (65)  applied  orthogonal  coordinate 
systems  to  find  a  set  of  m  orthogonal  signals  with  minimum 
Gabor  bandwidth. 

D2.  Channel  Models 

Channel  disturbances  and  channel  distortions  are  the 
basic  limitations  to  higher  data  rates.  Nonbinary  systems 
will  be  able  to  fulfil  their  promises  only  when  their  designers 
havb  the  tools  to  describe  the  channel  characteristics  with 


the  needed  precision.  In  sections  B2  and  B3  we  listed  all 
tho  adverse  effects  which  designers  have  to  consider.  Here 
we  review  the  large  effort,  going  on  at  present  and  still 
increasing,  which  ultimately  will  provide  convenient  mathe¬ 
matical  models  for  all  practical  channels. 

Again  it  should  be  stressed  that  the  model  of  a  linear 
distortion-free  channel  disturbed  only  by  additive  white 
Gaussian  noise  is  very  useful  as  a  reference  standard.  As  a 
first  approximation,  it  is  certainly  true  that  a  system  which 
excels  against  this  ideal  channel  model  will  also  perform 
well  against  other,  more  realistic,  channel  models.  For 
further  refinement,  a  system  should  be  modified  so  that  it  is 
best  matched  to  the  specific  channel  characteristics  at  hand. 
The  ideal  model  is  adequately  covered  in  many  textbooks, 
although  the  treatment  of  special  problems  like  the  “  axis 
crossing  ”  is  more  likely  to  be  found  in  journals  (Helstrom  57, 
McFadden  56  and  58). 

Elliott’s  (65)  paper  on  a  model  of  the  switched  telephone 
network  for  data  communications  may  show  us  how  well 
the  “  model  builders  ”  are  progressing.  This  example 
includes  a  short  historical  review  of  channel  modols  and 
demonstrates  how  the  application  of  realistic  channel  models 
simplifies  tho  analysis  of  error  control  systems  and  the 
determination  of  error  rates  for  error  control  codes.  The 
paper  also  indicates  the  complexity  of  such  models  by  using 
about  one  dozen  parameters  and  arranging  them  into  a 
mixture  of  three  distinct  channels.  Computer  simulation 
will  bo  unavoidable  when  applying  such  models  on  a  large 
scale. 

Time  varying  channel  characteristics  are  not  included  in 
Elliott’s  model,  but  they  are  very  important  for  radio  links. 
Bello  and  Nedlin  (62a,  b  ;  63,  64)  analyzed  the  performance 
of  selectively  fading  channels  when  used  for  binary  data 
transmission.  Turin  (58)  established  a  model  for  channels 
with  nonselective  slow  fading  and  referred  to  still  older 
contributions  to  models  of  general  multipath  channels 'r»s 
they  result  from  scatter  propagation  modes.  Reiger  (57) 
considered  still  earlier  a  model  of  a  fading  channel.  One 
Russian  paper  discussed  the  performance  of  a  reference 
signal  for  clocking  data  transmission  receivers  when  subjected 
to  fading  (Gerastovskiy  65)  ;  another  Russian  paper  used 
the  ratio  of  the  static  to  the  “  scattering  ”  power  as  the 
parameter  to  indicate  the  degree  of  fading  (Servinskiy  64). 
A  third  Russian  paper  by  Khvorostenko  (64)  investigated 
tho  performance  of  MPSK  systems  over  channels  with 
Rayleigh  fading,  one  of  the  most  frequently  used  models 
of  fading  channels.  Reports  of  U.S.  Government  Labora¬ 
tories  compared  the  theoretical  models  with  the  practical 
results  (Greim  et  al  65,  Korte  65).  This  leads  to  the  definition 
of  a  “  fading  bandwidth  ”,  a  term  which  may  find  more 
application  in  tho  future. 

Special  disturbances  have  been  investigated  extensively. 
Tho  most  general  models  are  those  which  consider  non- 
stationary  noise  components.  Belousov  (65)  investigated 
noise  characterized  by  the  product  of  a  Gaussian  stationary 
process  with  a  slow- varying  function  of  time.  Other  attempts 
to  generalize  the  mathematical  modols  of  external  distur¬ 
bances,  in  contrast  to  the  more  regular  internal  receiver 
noise,  led  to  models  which  assume  that  the  noise  characteris¬ 
tics  aro  only  partly  known  or  that  the  noise  is  correlated  to 
the  signals  (Nesteruk  and  Porfir’yeva  65,  Musa  63). 

Impulse  noise,  as  indicated  in  section  B2,  is  the  most 
critical  disturbance  in  telephone  circuits.  Numerous  in¬ 
vestigations  in  the  last  ten  years  provided  the  basis  for  a 
number  of  convenient  mathematical  models  which  describe 
approximately  the  characteristics  of  this  kind  of  noise. 
Mertz  (65),  in  a  recent  report  published  by  the  Rand  Corpora¬ 
tion,  summarized  the  results  as  they  were  available  early  in 
1965.  From  this  xeport  and  from  other  original  contributions 
one  learns  that  Gilbert  (60)  presented  a  simplo  on-off  model 
in  the  form  of  a  Markov  chain,  with  one  state  representing 
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error-free  transmission  and  another  state  representing  a 
transmission  period  with  constant  error  ratio.  In  1900 
Mertz  conceived  a  more  sophisticated  model  that  incorporated 
a  bi-modal  model  for  the  burst  duration.  One  mode  con¬ 
sisted  simply  of  1-bit  bursts;  the  other  mode  was  represented 
by  a  continuous  triangular  distribution  of  burst  durations. 
This  model  was  published  in  a  Rand  report  in  1900,  but  the 
report  was  released  in  1965.  Berger  and  Mandelbrot  (03) 
demonstrated,  with  the  aid  of  experimental  data  obtained 
from  the  German  telephone  network,  that  the  distribution 
of  inter-error  intervals  can  be  well  approximated  by  a  law 
of  Pareto  of  exponent  less  than  one.  Comparison  with 
experiments  led  Sussman  (63)  to  conclude  that  the  Pareto 
model  provides  an  excellent  representation  of  the  error 
statistics.  More  recently  Engol  (65)  used  a  model  consisting 
of  bursts  of  the  carrier  frequency  with  random  phase, 
occurring  randomly  in  time.  This  rather  simple  structure 
of  the  model  permitted  Engel  to  analyze  various  modulation 
systems  in  their  performance  under  such  noise  environment. 
Lindenlaub  (65)  claimed  that  a  still  simpler  model  consisting 
of  rectangular  single  pulses  of  constant  area  shows  good 
agreement  with  experiments. 

Models  of  atmospheric  noise  are  not  yet  as  numerous  as 
those  of  impulsive  noiso  in  telephone  circuits.  Reinmann 
(65)  used  a  distribution  for  atmospheric  noise  which  was 
originally  derived  in  a  paper  by  Beckmann  (64).  Reinmann’s 
thesis  contains  19  references  to  related  papers.  A  Japanese 
paper  investigated  the  crossing-rate  distribution  of  atmo¬ 
spheric  noise  envelopes  (Nakai  65),  and  Bello  (65b)  combined 
the  previously  discussed  fading  model  with  a  model  of 
atmospheric  noise,  thus  arriving  at  a  realistic  model  for  the 
comparison  with  actual  digital  communications  systems 
operating  over  HF  links. 

Errors  may  result  even  in  noise-free  transmission  circuits. 
Hill  (02)  established  mathematical  models  which  permit  the 
comparison  of  the  influence  of  distortions  with  the  influence 
of  noise.  Lukatela  (64)  showed,  in  a  German  paper,  that 
similar  models  may  bo  applied  to  account  for  the  influence 
of  delay  distortions. 

This  review  may  convince  the  reader  that  communications 
engineers  are  at  the  beginning  of  an  area  where  even  seemingly 
unpredictable  events  such  as  atmospherics,  fading,  and  other 
distortions  will  find  their  correct  place  in  mathematical 
models.  We  quoted  oidy  a  small  selection  of  all  the  publica¬ 
tions  in  the  area  of  communications  media  anti  their  charac¬ 
teristics.  We  attempted  to  select  only  those  references 
which  have  the  strongest  bearing  on  the  problems  of  non¬ 
binary  systems. 

D3.  Receiver  Models 

The  receiver  models  represent  the  lower  part  of  fig.  3 
marked  3c.  Mathematical  models  describing  the  performance 
of  a  data  receiver  may  range  from  general  models,  postulating 
basic  reception  theories,  to  specific  models,  describing 
individual  circuits  with  all  their  shortcomings.  Good 
examples  for  the  first  class  are  the  three  original  contributions 
by  Van  Meter  and  Middleton  (54)  and  Middleton  and  Van 
Meter  (55a,  b),  laying  the  foundations  of  modern  decision 
theory  for  communications  systems.  These  very  general 
models,  which  incorporate  all  classes  of  binary  and  nonbinary 
systems,  permit  the  derivation  of  general  rules  of  optimiza¬ 
tion.  These  models  cannot  bo  used  directly  to  predict  the 
performance  of  any  given  implementation  or  circuit.  For 
such  performance  predictions,  models  of  the  second  class 
are  needed.  A  good  example  is  a  paper  by  Karshin  (64) 
describing  a  mathematical  model  of  a  synchronous  digital 
receiver  (in  this  case,  for  binary  signals).  This  model 
permits  the  comparison  of  many  circuit  variations  and  can 
bo  used  to  predict  the  detailed  performance  of  any  one  of 
them. 

Apparently  both  classes  of  models  are  needed.  The  class 
of  general  models  has  progressed  to  some  degree  of  specializa- 
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tion.  Wyner’s  theory  of  bounded  discrepancy  decoding 
(Wyner  65),  for  example,  can  bo  applied  to  many  classes 
of  channels.  The  paper  itself  studies  the  theoretical  capa¬ 
bilities  of  minimum  distance  decoding  (MDD)  and  bounded 
distance  decoding  (BDD)  for  four  different  classes  of  com¬ 
munications  channels,  including  binary,  nonbinary,  and 
continuously  sampled  channels.  This  theory  soon  may  lead 
to  improved  decision  subsystems.  Kadota  (65)  reports  a 
theory  of  the  optimum  reception  of  m-ary  Gaussian  signals 
in  Gaussian  noise.  Although  the  title  suggests  a  further 
specialization,  the  theory  Is  virtually  a  generalization  by 
including  probabilistic  signals  (following  a  Gaussian  distribu¬ 
tion  similar  to  that  of  the  noise)  while  most  of  the  other 
theories  apply  to  “  sure  ”  signals.  Kadota  (64)  earlier 
developed  such  a  generalization  for  binary  Gaussian  signals. 
That  paper  contains  many  references  to  the  earlier  literature 
about  Gaussian  signals  (also  called  statistical  signals, 
probabilistic  signals,  stochastic  signals,  etc.).  The  decision 
process  deals  here  wit  h  the  discrimination  or  the  measurement 
of  statistical  parameters,  such  as  covariance,  moments,  and 
others.  Theories  of  this  nature  are  most  important  in  con¬ 
nection  with  unpredictably  time-varying  channels.  In 
such  channels,  not  only  the  noise  is  unpredictable  but  the 
signals  also  become  partly  unpredictable.  A  typical  channel 
with  such  characteristics  is  the  moon  bounce  relay.  Kasowski 
(63)  described  a  nonbinary  system  operating  with  noise-like 
signals  over  such  a  time-varying  channel. 

The  extractor  models  are  either  based  on  cross-correlation 
extractors  (Fano  51)  or  on  matched  filter  extractor  (Reiger  53, 
Turin  60).  The  former  variety,  indicated  in  fig.  3c,  is  more 
flexible.  The  mathematical  model  of  a  cross-correlation 
extractor  has  been  investigated  by  many  authors  (George  54, 
Lange  63,  Haas  61,  Wiorwillie  65,  Ludovici  63).  Most  of 
them  point  out  that  it  needs  still  further  improvement, 
primarily  when  applied  to  multidimensional  signals.  In¬ 
vestigations  of  the  output  of  multipliers  when  two  or  more 
random  variables  operate  at  the  input  (Lampard  56a, 
Donahue  64,  Springer  and  Thompson  64,  Lezin  65a,  b, 
Cooper  65)  may  be  helpful  in  this  direction.  Watanabe  (60) 
performed  an  information  theoretical  analysis  of  multi¬ 
variate  correlation  which,  though  it  aims  at  different  applica¬ 
tions,  may  be  of  importance  to  nonbinary  signal  extracting 
systems. 

The  decider  models  have  been  described  in  general  terms 
by  the  early  contributors  to  decision  theory  (Kotel’nikov  59a). 
Reiger  (58)  was  possibly  the  first  to  apply  the  maximum 
likelihood  decider  (MLD)  to  a  practical  model  of  a  nonbinary 
system  (MFSIC).  A  good  review  of  the  decision  subsystem 
for  equicorrelated  m-ary  signals,  together  with  many  refer¬ 
ences  to  earlier  contributions,  is  in  Nuttall’s  (62)  paper. 
Becker  et  al  (65)  compared  the  MLD  with  analog  demodula¬ 
tion  methods.  Other  contributions  to  the  theory  of  decision 
subsystems  are  available  in  papers  by  Rudnick  (61),  Blasbalg 
(61),  Muroga  et  al  (61),  and  by  Kohmenyuk  (63).  Two 
interesting  “  letters  to  the  editor  ”  are  concerned  with  very 
special  problems  of  the  “  largest-of  selection  systems  ” 
(Cooper  63)  and  the  Bayes  decision  procedure  (Ward  62). 
Hackett  (63)  derived  an  accurate  estimate  of  the  correlation 
performance  of  any  group  code  perturbed  by  white  Gaussian 
noise  and  compares  it  with  tho  performance  of  digital  decoding 
and  direct  transmissions  of  the  same  information.  He 
concludes  that  correlation  detection  offers  no  more  than 
3-db  power  contrast  advantage  over  digital  decoding. 
This  paper  assumes  that  in  each  case  a  MLD  is  applied  and  that, 
no  further  attempt  is  made  to  evaluate  any  runner-up 
decisions  information.  Indeed,  only  character  error  proba¬ 
bilities  were  evaluated  by  Hackett. 

The  decoder  models  are  usually  treated  in  connection  with 
receivers  for  coded  binary  transmissions.  Only  two  papers 
may  be  singled  out  for  references  in  connection  with  decision 
models.  One  is  Wyner’s  (65)  bounded  discrepancy  decoding, 
already  mentioned  in  coimection  with  the  discussion  of 
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general  receiver  models.  The  other  is  a  recent  Master’s 
thesis  (Wakeley,  .lr.  65).  This  thesis  forms  an  interesting 
bridge  between  the  general  statistical  sampling  theory  and 
the  sequential  decoding  theory  of  communications  systems. 
The  latter  theory  is  conventionally  applied  in  connection 
with  binary  decisions  and  will,  therefore,  not  be  discussed 
specifically  in  this  paper.  Wakeley ’s  thesis  Ls  much  broader 
in  its  potential  applications  to  nonbinary  systems  and  may 
he  of  great  interest  to  some  research  workers  on  nonbinarv 
decision  systems. 

Beyond  these  mathematical  models  of  extractors  and 
decoders,)  we  should  like  to  mention  a  number  of  receiver 
models  fdr  special  applications  which  do  not  lend  themselves 
easily  to  la  split  into  t  hese  two  kinds  of  subsystems. 

Polarity  coincidence  receiver  models  are  much  in  use  for 
radio  astronomy  receivers  (Ekre  63).  The  extraction  of  the 
epoch  of  overlapping  signals  was  the  subject  of  a  paper  by 
Young  (65).  The  extraction  of  signals  from  a  channel  with 
correlated  noise,  called  “  contrast  reception  ”  has  already 
been  mentioned  under  noise  models  (Nesteruk  and  Porfir’yeva 
65).  Daly  and  Rush  forth  (65)  discussed  the  non  parametric 
detection  of  a  signal  of  k  nown  form  in  additive  noise,  a  method 
which  may  gain  high  importance  for  the  detection  of  non- 
binary  signals  in  channels  with  non-Gaussian  noise.  Still 


Part  II— M 

Introduction 

Part  1 1  will  discuss  basic  approaches  for  denser  information 
packing .  Graphical  and  numerical  methods  developed  in 
Part  I  will  be  used  to  compare  these  various  approaches. 
A  basic  block  diagram  of  a  general  nonbinary  system  (fig.  3) 
will  sei-ye  as  the  common  link  when  explaining  the  differences 
between  the  various  methods.  All  nonbinary  systems 
presented  in  this  part  have  one  characteristic  in  common  : 
the  noitbinary  signals,  whatever  their  shape  and  mode  of 
generation  may  be,  are  transmitted  one  after  the  other,  in 
time  sequence.  The  receiver,  correspondingly,  will  make 
use  of  this  advance  information,  and  will  identify  in  each 
transmission  interval  one  ami  only  one  nonbinary  message 
(character). 

Sections  F,  G,  and  H  discuss  systems  that  respectively 
use  the  amplitude  (MASK),  the  frequency  (MFSlv),  or  the 
phase  j(MPSK)  of  a  sinusoidal  carrier  as  the  parameter  that 
assumes  one  of  the  permissible  states. 

E.  Fundamentals  of  Multistate  Systems 

The  basic  nonbinary  information  transmission  systems 
are  multistate  systems,  also  called  q-ary  (or  m-ary)  systems. 
These  terms  express  the  fact  that,  during  a  given  time  interval 
called  the  message  interval,  the  transmitter  assumes  one 
and  only  one  out  of  q  different  states.  All  q  states  are 
known  to  the  receiver  in  advance  of  any  transmission.  The 
most,  efficient  reception  method  will  be  selected  for  any 
case  at  hand,  making  use  of  this  a  priori  information.  Many 
different  methods  arc  known  for  the  specification  of  q  states 
of  a  transmitter  ;  the  various  sections  of  this  part  will 
describe  the  three  basic  q-ary  nonbinary  transmission 
methods. 

The  simplest  waveform  is  one  that  keeps  all  its  parameters 
constant  over  the  whole  transmission  interval  and  then 
changes  one  of  the  parameters  instantaneously  to  a  new 
value  when  a  new  transmission  interval  begins  which  carries  a 
new  information  character.  This  waveform  is  a  rectangular 
pulse  and,  in  its  most  general  form,  can  be  represented  by  a 
rectangular  carrier  pulse.  Figure  6  shows  that  the  pulse 
has  six  independent,  parameters,  three  of  which  are  normally 
used  (amplitude,  carrier  frequency,  carrier  phase)  while  a 


more  general  may  be  the  receiver  model  of  Taylor  (66) 
which  operates  with  a  nonlinear  time-invariant  filter  for  the 
detection  of  signal  pulses  of  known  shape  imbedded  in  non- 
stationary  noise.  The  problem  of  distribution -free  statistical 
detection  is  discussed  in  a  very  general  form  by  Bell  (64) 
in  a  U.S.  Government  report  which  is  available  to  the  public. 
These  methods  of  detection  can  be  applied  whenever  the 
underlying  signal  and  noise  distributions  arc  unknown. 
Although  these  ideas  originated  in  the  area  of  RADAR 
and  SONAR  detection,  they  may  have  a  deep  impact  as 
detectors  of  nonbinarv  signals  in  an  unpredictable  noise 
environment.  Adaptive  receivers  are  another  method  of 
dealing  with  unpredictable  channel  characteristics.  Many 
publications  have  been  issued  in  the  general  area  of  adapt  ive 
communications  systems.  We  restrict  our'  review  lo  the 
paper  of  Levin  and  Bonch-Bruyevich  (65)  which  discusses  a 
self-optimizing  decision  system  and  to  the  paper  of  Birdsall 
and  Roberts  (65)  on  an  optimum  nonsequential  observation- 
decision  procedure.  Both  papers  seem  to  open  some  new 
avenues  for  the  receiving  methods  in  nonbinary  systems. 

Summarizing,  wo  may  point  to  the  large  variety  of  receiver 
models  which  are  available  for  nonbinary  receiving  systems. 
It  will  be  shown  in  the  next  Parts  how  a  number  of  these 
models  arc  already  in  use  for  the  analysis  of  special  nonbinary 
systems. 


I ti state  Systems 

fourth,  the  epoch  t,  (i.e.,  I  he  position  of  a  signal  within  a 
message  interval),  is  Occasionally  used  in  multiple  time  shift 
keying  (MTSK)  systems.  The  fifth  parameter,  the  duration 
of  a  waveform,  may  likewise  be  used  in  MTSK  systems, 
lint  will  normally  he  fixed  by  the  input  rate  and  the  number 
of  subintervals  in  a  message  interval.  The  sixth,  the  tic 
component,  could  be  used  in  certain  channels  to  carry 
information,  but  in  most  situations  it  may  suffer  severe 
distortions  or  it  may  interfere  with  other  service  functions 
of  the  system,  such  as  synchronizat  ion  or  Doppler  compensa¬ 
tion  (carrier  acquisition). 

In  the  language  of  the  block  diagram  of  a  general  nonbinarv 
system  (fig.  3),  we  can  specify  for  till  systems  of  Part,  II 
t  he  following  set  of  condit  ions  : 

(i)  The  input  will  bo  binary. 

(ii)  The  S/P  translator  will  be  necessary  for  all  nonbinarv 
systems. 

The  encoder  will  simply  combine  n  binary  input  elements  to 
one  2"-arv  internal  trigger  signal  that  will  go  to  the  “  hold  ’’ 
circuit.  This  may  go  as  a  multilevel  signal  over  a  single 
line  7  or  by  a  binary  trigger  signal  over  one  out  of  2"  lines 
(multiple  frequency  shift  keying  [MFSK])  or  in  any  other 
convenient  way.  The  modulating  matrix  and  the  signal 
generator  may  assume  several  forms,  as  will  bo  discussed 
in  the  following  sections.  At  the  receiving  side,  there  will 
be  a  similar  variety  of  approaches  for  the  arrangement  of  the 
carrier  extractor,  the  signal  generator,  and  the  demodulating 
matrix.  The  hold  and  integrate  circuit  will  be,  in  most 
systems,  a  single  resampler,  a  comparator,  or  a  summing 
network.  The  decider  will  be  a  simple  single  or  multiple 
threshold  device  sending  trigger  signals  to  the  "hold  and 
P/S  ”  device,  which  will  finally  restore  the  continuous 
binary  serial  output  information  (low.  No  direct  influence 
of  a  return  channel  will  be  considered,  although  some  of  the 
practical  systems  plotted  in  the  utility  charts  do  employ 
communications  feedback  over  a  return  channel. 

Expressing  the  preceding  assumptions  in  terms  of  tig.  4, 
it  can  be  seen  that  these  assumptions  imply  digit -by-digit 
transmission,  the  only  mode  considered  in  Part  11.  In  fig.  4 
the  digit  B'  transmitted  by  the  choice  of  frequency  f,  will 
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f(t)  =  b  +  o  sin  (|2  t  +  4>c)  ■  •  •  t,  <  t  <  t2 
f(t)  =  0 . Kfj  ;  t>t2 


a  •  •  amplitude  of  carrier 

b  -  -  dc  component 

•  •  •  epoch,  beginning  of  waveform 
t2  •  •  •  end  of  waveform:  ^2  =  M 

tc  •  •  •  period  of  carrier:  l/rc  =  fc  carrier  frequency 

♦  •  •  •  carrier  phase:  Ta  =  — —  •  T 

2n  c 

Figure  0. — The  six  parameters  of  a  typical  rectangular  carrier 

signal. 


lie  independent  of  the  preceding  digit  A'  anil  the  following 
digit  C'.  The  extractor  and  the  decider  will  consider  the 
data  for  each  digit  separately  and  there  will  he  no  correlation 
of  the  extractor  output  over  several  digits.  This  restriction, 
together  with  the  previously  mentioned  restriction  that  only 
one  parameter  will  be  shifted  by  the  transmitter  and  that 
all  other  parameters  will  be  held  constant,  characterizes 
all  systems  of  Part  II  as  one-dimensional  binary  or  nonbinary 
systems.  This  statement  holds,  notwithstanding  the  fact 
that  certain  MPSIv  systems  generate  their  signals  in  a 
t  wo-dimensional  way  by  superposing  two  phase-quadrature 
components.  They  still  result  in  a  one-dimensional  trans¬ 
mission  signal  as  long  as  they  operate  under  the  restraint 
of  a  constant  amplit  ude.  A  truly  two-dimensional  generator 
must  produce  output  signals  with  two  degrees  of  freedom 
(in  phase  and  in  amplitude).  Part  1 1  f  will  include  discussions 
of  such  multi-dimensional  syst  ems  as  well  as  serially  encoded 
systems,  which  introduce  purposely  an  interdependence  of 
consecutive  digits,  even  if  all  of  them  are  carrying  the  in¬ 
formation  in  the  same  and  only  parameter.  As  explained 
in  Part  I,  in  that  case,  each  digit  is  a  new  dimension  of  the 
total  message  (or  word). 

Binary  systems  will  be  frequently  considered,  either  as  the 
basis  of  nonbinary  systems  or  for  the  purpose  of  showing 
the  improvement  of  the  nonbinary  system  over  the  binary. 
In  several  cases  it  will  be  necessary  to  refer  to  systems 
which  encode  the  information  into  several  subchannels  that 
carry  their  signals  simultaneously  (in  frequency  division) 
over  the  same  channel  (for  example,  Kineplex  in  section  Id). 
Yet,  as  explained  before,  the  purpose  of  this  paper  does  not 
include  a  complete  survey  of  poly-signal  systems. 

F.  Multiamplitude  Shift  Keying  (MASK)  Systems 

FI.  Ideal  MASK  Systems 

Binary  systems  inside  computers  and  logical  devices 
operate  with  rectangular  signals,  clearly  the  simplest  signal 
waveform  that  anyone  could  use  in  multiamplitude  shift 
keying  (MASK)  systems.  Waveforms  of  this  kind  can  be 
used  in  unipolar  or  bipolar  mode.  Figure  7a  and  b  show 
examples  of  rectangular  MASK  waveforms  with  V«  being 
t  he  peak-to-peak  value  in  either  case. 

Assuming  an  ideal  low-pass  channel — a  channel  with  a 
rectangular  frequency  response  from  dc  to  a  delinito  cutoff 


frequency  f„  and  zero  response  for  frequencies  beyond  f„ — 
leads  to  the  sin  x  over  x  waveforms  shown  in  fig.  7c.  These 
waveforms,  which  were  used  in  the  early  theory  of  MASK 
systems,  were  assumed  when  plotting  the  ut  ility  curve  (No.  2) 
for  an  ideal  low-pass  MASK  system  (fig.  1).  This  curve 
derives  from  Oliver  et  al  (48)  ;  also  appearing  as  the  upper 
bound  for  an  ideal  MASK  system  in  fig.  8,  it  is  drawn  for  a 
binary  error  ratio  at  the  receiver  output  of  It)-3.  This  ideal 
MASK  system  may  be  considered  a  special  case  of  the  general 
nonbinary  system  of  fig.  3,  with  the  waveform  alphabet, 
consisting  of  q  signals,  all  of  equal  shape,  but  with  the  ampli¬ 
tudes  equally  spaced  between  the  negative  peak  value  and 
the  positive  peak  value  (including  the  peak  values).  Figure  7d 
show's  an  example  of  a  message  formed  by  this  bipolar 
system.  The  utility  curve  of  an  ideal  system  of  this  kind 
(No.  2,  fig.  8)  is  based  on  the  assumption  of  a  linear,  distortion- 
free  low-pass  channel  with  additive  white  Gaussian  noise. 
The  demodulating  matrix  of  fig.  3  consists  in  this  case  simply' 
of  a  precisely  timed  resampler,  and  the  decoder  is  merely  a 
digit-by-digit  decider  with  thresholds  exactly  halfway 
between  any  two  amplitude  levels. 

Another  ideal  system  is  presented  by  Arthurs  and  Dym  (02) 
on  the  assumption  that  the  unipolar  (also  called  monopolar) 
rectangular  waveforms  of  fig.  7a  form  the  rectangular 
envelopes  of  a  cosine  carrier  waveform  of  a  frequency  at 
the  centre  of  the  transmission  band.  The  authors  calculated 
the  character  error  ratio  of  this  sy'stem,  assuming  a  linear 
distortion-free  bandpass  channel  with  white  bandlimited 
Gaussian  noise  and  either  a  coherent  ideal  matched  filter 
extractor  and  demodulator  or  a  noncoherent  demodulator. 
The  decider  operates  again  with  thresholds  halfway  between 
the  equally'  spaced  amplitude  levels.  We  used  Arthurs 
and  Dymi’s  curves  and  converted  them  to  the  magnitudes 
used  in  our  utility  chart  (fig.  8a).  For  the  coherent  mono- 
polar  MASK  system,  this  leads  to  the  equation  : 


<•« 


— .  q{  .  / _ _ } 

logs  d  IV  u  (q  -  ])  (2q  -  I)J 


(20) 


In  translating  Arthurs  and  Dym’s  equation  59,  notice 
t  hat  their  error  probability  Pe  is  a  character  error  probability. 
Assuming  a  low  error  ratio  (single  errors  only)  and  assuming 
the  application  of  a  Gray'  code,  we  translated  P„  into  cb 
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Figure  7. —  Unipolar  ami  bipolar  low-pass  signals. 


Figure  8. — (a)  Utility  curves  of  MASK  systems, 
(b)  Error  function. 


according  to  our  lower  bound  explained  in  Part  I  (equations  2 
and  3).  Tho  number  of  levels  (q  in  our  ease)  has  the  same 
meaning  as  m  in  the  Arthurs  and  Dym  paper.  The  integral 
is  tabulated  in  most  mathematical  table  books.  For  the 
benefit  of  our  readers,  we  plotted  it  as  Q(x)  in  fig.  8b.  Notice 
thaf  this  is  related  to  the  frequently  used  integral  P(x)  by 
the  relation  : 


Q{x}  =  1  -  P{x}  ;  P{x}  =  —  f  e-*2/2  dz  (27) 

VTuJ-“ 

It  can  be  seen  that  equation  26  is  actually  tho  equation 
for  an  error  probability  curve  as  discussed  in  Part  I,  section  C3. 
To  bring  it  in  line  with  our  notation,  we  also  had  to  modify 
the  energy  contrast,  which  Arthurs  and  Dym  referred  to  as 
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the  energy  per  character  E.  We  defined  the  energy  contrast 
on  the  basis  of  the  energy  per  output  bit  of  information.  As 
long  as  there  Is  no  redundancy  in  the  transmission  (syn¬ 
chronization  characters  etc.)  and  no  coding  involved,  the 
relationship  between  character  and  bit  energy  contrast  is 
simple  : 

ce  =  cB  ■  log.  q  (28) 

Going  from  the  error  probability  curve  to  the  utility  curve 
is  simple  when  one  remembers  that  the  utility  Is  the  reciprocal 
value  of  the  bit  energy  contrast  (Part  I,  section  C2).  But 
one  has  not  yet  introduced  the  bit  density,  which  requires  a 
knowledge  of  the  signal  base.  We  recall  from  Part  I, 
section  C3,  that  the  bit  density  is  defined  as  the  ratio  of  the 
transmission  rate  R/j  in  bits  per  second  to  the  transmission 
bandwidth  Bra  .  Introducing  the  duration  of  one  character 
T c  and  assuming  again  that  there  is  no  redundancy,  one  can 
also  express  the  bit  density  in  the  form  : 


Bb  =  log,  q 
Btr  Tc-Btb 


(29) 


Arthurs  and  Dym  suggest  in  section  G  of  their  paper  a 
value  of  1.5  for  the  signal  base  (TB  product)  of  each  character. 
This  gives  us  the  final  equation  for  relating  the  alphabet 
size  q  to  the  bit  density  : 


D  = 


i°gj  q 
1.5 


2 

j  l°gj  C1  • 


(30) 


D«  =  1.75  +  10  logjQ  (log.  q)  .  (30a) 


It  is  now  possible  to  plot  the  utility  curve  of  Arthurs 
and  Dvm’s  ideal  monopolar  carrier-band  MASK  system. 
This  is  done  in  fig.  8a  by  curve  4.  Remember  that  we  plot 
utility  curves  in  a  logarithmic  scale,  i.e.,  both  axes  are 
calibrated  in  decibels.  When  plotting  curve  4,  one  should 
recall  that  a  utility  curve  is  only  valid  for  a  fixed  parameter 
of  the  error  ratio.  In  the  case  of  curve  4,  we  use  e«  =  1 0  -  * 
as  the  constant  value  which  goes  to  the  left  side  of  equat  ion  26. 
This  equation  must  then  be  solved  for  u,  using  different 
values  for  q  and  taking  the  values  of  the  function  Q(x)  from 
fig.  8b.  Expressing  the  utility  (u)  in  decibels  (db)  gives 
the  ordinate  value  for  each  point  of  the  curve.  The  corres¬ 
ponding  D  value  (abscissa)  residts  from  equation  30a  for 
each  q. 

Similarly,  Arthurs  and  Dym’s  equation  107  may  be  con¬ 
verted  to  the  magnitudes  used  in  this  paper.  That  equation 
gives  an  upper  bound  and  a  lower  bound  for  the  error 
probability  of  a  monopolar  MASK  system  with  incoherent 
detection.  The  converted  formula  yields  : 

B(q)  logio  {es  log.  q)  <  u  <  B(q)  log,„  {eB  log.  q  +  log10  q)  ,  (31) 


B(q)  = 


1.535  .  (q  -  I)  .  (2q  -  1) 
log2q 


(31a) 


Notice  that  equation  31  gives  the  value  for  u.  To  be 
plotted  in  fig.  8a,  it  must  be  contorted  to  decibels.  The 
range  between  upper  and  lower  bound  is  plotted  as  area  5 
in  fig.  8a.  To  get  the  corresponding  values  for  D  in  db, 
one  must  use  equation  30a,  the  same  as  in  the  previous  case 
of  coherent  monopolar  MASK. 

A  discussion  of  the  questions  raised  by  a  comparison  of 
these  three  ideal  MASK  systems  may  be  useful  to  the  reader. 
Curve  2  in  fig.  8b  represents  an  ideal  low-pass  system, 
having  extremely  low  frequency  components  and  using  the 
tightest  packing  of  waveforms  one  can  imagine.  Actually 
the  composite  of  waveforms  formed  by  a  sequence  of  MASK 
signals  according  to  this  ideal  system  is  an  analog  waveform  ; 
and  no  practical  system,  so  far,  has  been  devised  to  code 
input  information  into  such  a  composite  transmission  signal 
without  distortions.  No  practical  channels  are  known  that 
would  handle  such  waveforms  without  distortions.  Thus 
one  may  expect  that  all  practical  MASK  systems  will  have  the 
utility  curves  below  curve  2. 


The  ideal  carrier-band  MASK  systems  of  Arthurs  and 
Dym  use  waveforms  that  are  easy  to  generate,  i.e.,  rectangular 
transmission  signals.  This,  at  the  same  time.  Is  why  curves  4 
and  5  are  shifted  so  far  to  the  left  side  of  fig.  8a.  Indeed, 
for  using  rectangular  transmission  signals,  the  authors 
pay  a  penalty  of  1  :  3  ( —5.67  db)  in  bit  density  against  the 
low-pass  system  of  curve  2,  which  operates  at  the  Nyqulst 
rate  (/?0  =  0.5).  Even  with  this  penalty,  it  is  not  reasonable 
to  expect  that  no  distortions  would  be  inflicted  upon  the 
rectangular  pulses.  For  example,  a  special  message  sequence 
would  cause  the  transmitted  waveform  to  alternate  between 
maximum  level  and  zero  level,  causing  a  sin  x  over  x  spectrum 
that  extends  far  beyond  the  cutoff  bandwidth  of  1.5Te  . 
The  absence  of  these  portions  of  signal  energy  which  cannot 
be  transmitted  causes  ringing  in  the  receiver  waveform  that 
may  easily  exceed  the  threshold  of  the  first  level  in  a  system 
with  a  high  q  value.  The  next  subsection  will  review  a 
number  of  research  efforts  aiming  at  more  suitable  waveforms 
for  a  one-dimensional  MASK  system  than  sin  x  over  x  or 
rectangular  signals.  Part  IV  of  the  paper  will  show  how 
bandlimited  orthogonal  signal  alphabets  are  still  better 
suited  to  solve  this  problem  for  multidimensional  MASK 
systems. 

Another  puzzling  observation,  when  comparing  the  three 
ideal  MASK  systems,  may  be  the  small  difference  between 
coherent  and  noncoherent  MASK  systems.  Curve  4  repre¬ 
sents  a  coherent  monopolar  system — a  highly  redundant 
system,  which  does  not  make  use  of  its  redundancy.  Once 
coherent  reception  can  be  secured,  one  may  as  well  use  all 
the  negative  amplitudes.  Indeed,  under  assumption  of  a 
monopolar  waveform  alphabet,  there  is  not  much  advantage 
in  a  coherent  receiver  over  an  envelope  detector,  although 
their  analytical  treatment  is  still  considerably  different,  as 
shown  by  the  different  shapes  of  equations  26  and  31.  It 
was  the  intention  of  the  paper  by  Arthurs  and  Dym  to  point 
out  this  fact.  Truly  bipolar  MASK  systems  will  be  discussed 
in  subsection  F6. 

Yet  another  interesting  observation  Is  the  fact  that  all 
these  systems  have  their  operating  point  for  a  given  q 
value  at  about  the  same  power  contrast,  (SNR)  represented 
by  the  dashed  sigma  lines.  Another  very  instructive 
comparison  results  when  using  equation  26  for  the  special 
case  q  =  2,  representing  a  coherent  binary  AM  system  : 

eB  =  Q{\^}  =  Q{V/i}.  (32) 

This  equation  was  originally  derived  by  Reiger  (53), 
Helstrom  (55),  and  Lawton  (58)  who  arrive  at  a  general 
equation  for  coherent  binary  ideal  systems  introducing  the 
correlation  coefficient  A  between  the  two  waveforms  repre¬ 
senting  Mark  and  Space  respectively. 

Considering  further  that  equation  26  holds  for  unilateral 
MASK  systems  with  zero  amplitude  level  as  the  lowest 
level,  we  recognize  that  this  zero  amplitude  level  corresponds 
to  A  =  0  in  Lawton’s  ideal  system.  Thus  equation  26  leads, 
in  the  limiting  case  of  q  =  2,  to  results  identical  with  Lawton’s 
ideal  binary  system.  The  fact  that  Lawton’s  system  is 
derived  for  two  waveforms  of  equul  energy,  while  the  MASK 
system  of  equation  26  uses  one  waveform  of  zero  energy, 
does  not  matter  because  E  in  equation  26  is  the  average 
energy  of  all  equally  probable  waveforms.  Naturally,  the 
highest  utility  for  any  binary  system  can  bo  reached  if  the 
two  waveforms  have  the  maximum  cross-correlation  coeffi¬ 
cient  of  A  =  —1,  which  they  reach  if  they  are  the  negatives 
of  each  other.  This  leads  to  the  equation  for  the  best 
possible  binary  system,  a  phase-coherent  system  with  ident  ical 
waveforms  of  opposite  polarity  using  a  matched  filter 
extractor. 


The  utility  of  this  optimum  ideal  binary  system  depends 


December,  1969 


Proceedings  I.R.E.E.  Australia  397 


F I L I PO W SKY  :  Nonbinary  Information  Transmission 


only  upon  the  acceptable  eiTor  ratio.  The  utility  can  be 
read  off  directly  from  fig.  8b  for  any  given  error  ratio.  Assum¬ 
ing  c b  -  10-s  we  find  x  =  3.10  ;  u-1  —  4.81  or  u  =  —0.8  db. 
This  value  is  plotted  as  the  utility  curve  3  of  the  ideal  binary 
system  in  fig.  8a  and  also  in  fig.  1  of  Part  I  as  curve  3.  It 
joins  the  ideal  low-pass  MASK  system  at  the  Nyquist  rate 
(D  =  3  db).  Curve  3  is  a  straight  line,  demonstrating  that 
the  utility  of  the  ideal  binary  system  cannot  be  raised  by 
reducing  the  bit  density.  Only  nonbinary  systems  have  tho 
capability  to  trade  bit  density  for  utility.  The  binary 
system,  however,  can  trade  off  power  contrast  versus  trans¬ 
mission  rate  as  mentioned  in  seotion  C2  in  Part  1  when 
explaining  the  utility  chart. 

F2.  Waveform  Design  in  MASK  Systems  for  Low-Pass 
Channels 

The  comparison  of  the  three  ideal  MASK  systems  clearly 
showed  the  need  of  a  more  sophisticated  waveform  design. 
Indeed,  many  researchers  tinned  their  attention  to  tbis 
problem1.  We  shall  first  discuss  the  results  for  low-pass 
channels. 

Gilbert  (52)  had  already  investigated  many  nonbinary 
alphabets.  One  of  those  alphabets  has  three  levels  (PA  1 
in  Gilbert’s  fig.  2)  ;  another  one  has  five  levels  (PA  2). 
Gilbert  refers  to  these  waveform  alphabets  as  quantized 
pulse  amplitude  modulation.  The  two  operating  points  are 
placed  into  fig.  8a  as  points  0  and  7  respectively. 

Hill  (62)  performed  an  intensive  investigation  of  the 
influence  of  waveform  distortions  on  the  maximum  number 
of  amplitude  levels  that  can  be  discriminated  without  error 
in  a  poise-free  low-pass  channel.  He  based  his  work  on 
the  theoretical  fundamentals  of  pulse  transmission  published 
by  Suride  (54).  Hill  defines  a  maximum  error-free  signalling 
rate  (m.e.s.)  and  shows  that  this  magnitude  depends  in  a 
complicated  way  on  the  number  of  levels  used  for  the  wave¬ 
form  alphabet  of  a  MASK  system.  Examples  are  given  in 
which  m.e.s.  increases  without  limit ;  passes  through  a 
peak  ;  remains  constant  or  tends  asymptotically  to  a  constant 
value  :  or  decreases  steadily,  as  the  number  of  transmitted 
levels  (increases.  Hill  concludes  that  in  a  noise-free  situation 
there  I  is  no  large  difference,  whether  one  tries  to  increase 
the  bit  density  by  packing  binary  digits  more  densely  than 
the  Nyquist  rate  and  permitting  them  to  overlap,  or  to  keep 
the  digital  rate  below  the  Nyquist  rate  and  pack  more  informa¬ 
tion  into  each  digit  by  permitting  more  than  two  levels. 
In  either  case  the  designer  learns  to  tolerate  the  resulting 
intersymbol  distortions. 

Lord  and  Lytle  (64),  recognizing  that  the  ideal  low-pass 
filter)  is  unrealizable,  nonetheless  use  it  as  a  guide  to  novel 
techniques  to  solve  the  signal  packing  problems  for  low-pass 
channels.  One  of  these  is  tho  pseudo-sample  insertion  (PSI) 
technique,  which  is  actually  designed  to  reduce  the  so-called 
aliasing  error  in  sampled  pulse  transmission  systems.  In 
modified  form  it  could  be  applied  in  MASK  systems.  A 
second  technique  suggested  by  Lord  and  Lytle  is  known 
as  the  vestigial  interpolation  filter  (VEF)  technique.  Rather 
than  attempting  to  eliminate  the  alias  energy,  this  method 
makes  use  of  it.  It  achieves  the  spectrum  limitation  in 
the  frequency  domain,  while  the  PSI  technique  achieves 
the | same  goal  in  the  time  domain.  The  VI F  technique  is, 
therefore,  closely  related  to  the  various  vestigial  sideband 
techniques  of  MASK  transmission  that  we  shall  review  in 
subsection  F5. 

Another  group  of  research  workers  operates  on  the  basis 
that  the  whole  system,  transmitter-channel-receiver,  must 
be  |  optimized  to  achieve  the  highest  information  density. 
A  good  guide  for  this  approach  is  the  paper  by  Tufts  (65) 
which  gives  38  references  to  earlier  papers.  Gerst  and 
Diamond  (61)  demonstrated  how  conditions  on  the  input 
waveform  to  a  given  system  can  be  formulated  to  eliminate 
completely  the  output  tail  which  normally  causes  the  inter- 
syinbol  interference.  Petrovich  and  Razmakhnin  (65)  and 


Cory  (66)  investigated  the  distribution  of  the  waveform 
energy  in  multilevel  signalling  systems. 

F3.  Nonbinary  Pulse  Code  Modulation  Systems 

One  important  application  of  MASK  methods  is  the  pulse 
code  modulation  (PCM)  systems.  PCM  was  invented  by 
A.  H.  Reeves  (42)  more  than  25  years  ago.  It  was  intended 
to  improve  communications  in  two  aspects  :  the  first  one 
is  the  noise  immunity  afforded  by  a  digital  system,  as  com¬ 
pared  with  an  analog  system  (Goodall  47)  ;  the  second  aspect 
is  the  economy  in  equipment  when  combining  PCM  with 
synchronous  time-division  multiplexing  as  compared  to 
frequency  division  multiplexing  of  many  analog  channels 
(Bennett  41).  A  third  aspect,  not  suggested  by  the  original 
inventor  but  frequently  proposed  since,  is  the  possibility  of 
achieving  more  efficient  information  packing  through 
asynchronous  multiplexing  (Filipowsky  57).  PCM  is  gaining 
increasing  importance  in  modern  short-haul  (Davis  62)  and 
long-distance  (Litchman  63)  communications  trunks. 

Ternary  PCM  is  the  most  logical  step  to  anticipate  when 
extending  binary  PCM  to  multilevel  PCM.  In  the  passive 
ternary  form  there  are  three  states  :  plus  peak  value,  zero, 
and  minus  peak  value.  This  passive  ternary  inode  of  opera¬ 
tion  has  many  interesting  applications  (Filipowsky  and 
Scherer  56).  It  could  be  used  to  increase  the  bit  density 
by  a  factor  of  log.  3  or  for  +1.59  db.  There  are  many  ways 
to  organize  the  encoder  of  fig.  3  for  translating  the  binary 
input  into  passive  ternary  waveforms  of  equal  rate  so  that 
the  transmission  signal  has  no  dc  component  and  will  meet  a 
number  of  other  requirements  which  differ  from  case  to  case 
(Aaron  62b).  One  possibility  is  time  polarity  control  (TPC), 
first  suggested  by  L.  C.  Thomas  of  the  Boll  Telephone 
Laboratories  (BTL),  where  timeslots  arc  labelled  alternately 
positive  and  negative.  If  a  unipolar  pulse  representing  the 
binary  one  occurs  in  a  positively  labelled  timeslot,  it  is 
transmitted  unaltered.  On  the  other  hand,  if  a  binary  one 
occurs  in  a  timeslot  with  a  negative  label,  the  pulse  is  trans¬ 
mitted  with  negative  polarity.  Zeros  in  the  binary  input 
are  unaffected  and  will  be  transmitted  as  zeros  in  the  passive 
ternary  train.  Such  signals  produce  a  power  spectrum  with 
discrete  components  at  odd  integer  multiples  of  half  the  bit 
rate  (Aaron  62a). 

The  interleaved  bipolar  operation,  invented  by  M.  Karnaugh, 
leads  to  still  another  restricted  ternary  mode  (Aaron  62b). 
It  is  a  combination  of  TPC  with  the  bipolar  mode  and  produces 
nulls  at  the  integer  multiples  of  a  fraction  of  the  bit  rate. 
These  interleaving  modes  of  operation  are  particularly 
important  in  the  short-haul  applications  of  PCM,  where  it  is 
necessary  to  use  many  bipolar  repeaters  in  tandem  (Mayo  62) 
and  where  the  nonideal  circuits  and  terminals  in  the  exchange 
plant  produce  particularly  severe  limitations  to  the  waveforms 
that  can  be  handled  (Shennum  and  Gray  62). 

More  recently,  Sipress  (65)  of  Bell  Telephone  Laboratories 
discussed  a  further  improvement  of  tho  previously  mentioned 
ternary  modes,  which  he  calls  the  paired  selected  ternary 
(PST)  transmission  plan.  This  plan  improves  the  trans¬ 
mission  of  the  timing  information,  permits  monitoring  of 
the  lines  during  operation,  and  retains  the  elimination  of  the 
dc  component  in  the  same  way  as  the  other  ternary  coding 
procedures.  It  takes  pairs  of  binary  input  digits  and  trans¬ 
lates  them  into  pairs  of  ternary  transmission  digits,  thus 
offering  a  redundancy  of  5  over  6  (more  than  50  percent), 
which  can  be  used  in  a  variety  of  submodes. 

Another  extension  of  the  bipolar  codes  to  higher  degrees 
is  reported  by  Yazaki  et  al  (65).  These  authors  investigated  a 
number  of  PCM  translation  procedures  under  the  names  of 
“  pseudo-ternary  code  ”  and  “  bimode  code  ”. 

While  the  previously  mentioned  binary-to-ternary  codes 
inserted  redundancy  to  solve  certain  transmission  problems, 
research  also  continues  in  the  opposite  direction  (i.e.,  reducing 
the  redundancy  contained  in  the  input  information).  From 
the  several  hundred  publications  in  the  field  of  redundancy 
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reduction  (or  information  compression),  we  ment  ion  here  only 
two  that  are  directly  related  to  nonbinary  PCM  :  Farnqui 
and  Das  (04)  reported  tests  of  a  “  slope-quantized  ternary  ” 
PC'M  system  that  operates  in  a  range  from  25-  to  35-db 
power  contrast  with  a  pulse  repetition  rate  of  20  to  60  KHz. 
Deregnancourt  (65,  60)  introduced  the  concept  of  variable 
length  words  for  PCM  transmission.  It  is  a  combination  of 
time  assignment  speech  interpolation  (TASI)  and  conven¬ 
tional  PCM.  Basically  it  avoids  the  transmission  of  all 
nonsignificant  zeros  in  the  PCM  words. 

F4.  Special  MASK  Low-pass  Systems 

In  this  subsection  we  shall  discuss  a  further  extension 
of  the  idea  of  translating  the  information  from  binary  to 
nonbinary  alphabets.  The  goal  of  the  research  reported 
below  is  the  improved  matching  of  the  transmission  signals 
to  the  channel  characteristics,  attacking  the  signal-shaping 
problem  with  the  help  of  special  code  translation  rules  when 
going  from  binary  to  nonbinary  codes.  A  first  attempt  in 
this  direction  is  the  binary  to  ternary  code  conversion 
(subsection  F3).  It  suppresses  the  de  component.  In 
t  he  following  paragraphs,  the  goal  is  the  creation  of  a  specially 
shaped  power  spectrum  of  the  transmission  signals,  matching 
closely  the  frequency  response  of  a  telephone  circuit  or  a 
general  cosine-square-shaped  frequency  response. 

The  most  publicized  efforts  in  this  direction  are  the  research 
efforts  of  Lenkurt  Electric  Company,  represented  mainly 
by  the  paper’s  of  Lender.  The  duobinary  technique  (Lender 
03a,  b)  Ls  the  first  step  to  a  more  general  polybinary  concept 
(Lender  04a,  b)  or  correlative  level  coding  for  binary  data 
transmission  (Lender  60).  The  basic  idea  is  that  of  employing 
an  encoder  with  memory,  including  the  last  K  binary  input 
digit  in  the  encoding  process  for  each  new  q-ary  transmission 
digit.  In  the  duobinary  mode  (K  2)  the  last  two  binary 
input  digits  will  be  considered  when  forming  the  next  ternary 
(q  3)  transmission  digit.  The  digital  rale  of  the  binary 
input  and  the  ternary  transmission  signal  is  exactly  the  same. 
This  allows  redundancy  for  the  transmission  waveforms. 
This  redundancy,  according  to  Lender’s  proposal,  is  used 
to  shape  the  transmission  waveforms  in  a  way  that  they 
concentrate  their  energy  into  the  lower  frequencies,  still 
avoiding,  if  required,  any  de  component.  To  achieve  this 
goal  he  imposed  on  the  encoder  a  translation  rule  that 
never  permits  a  change  in  the  q-ary  levels  of  more  t  han  one 
step  when  going  in  time  sequence  from  one  q-ary  digit  to 
the  next.  This  transmission  role  makes  sharp  transitions 
impossible  in  the  final  transmission  waveform  of  the  MASK 
signal,  thus  avoiding  strong  energy  in  the  high  frequency 
end  of  the  spectrum.  Alternately,  if  this  high  frequency 
energy  can  be  tolerated,  the  digital  transmission  rate  can 
be  raised,  increasing  the  bit  density  of  the  system.  Even 
with  this  restrictive  translation  rule,  a  certain  amount  of 
redundancy  still  remains  in  the  q-ary  transmission  sequences, 
which  can  be  used  for  a  limited  amount  of  error  detection. 

’Phis  correlative  level  coding  method  is  specifically  designed 
for  non-ideal  transmission  channels.  Its  efficiency,  accord¬ 
ingly,  depends  on  the  degree  of  matching  between  the 
resulting  transmission  waveforms  anil  the  actual  channel 
characteristics.  This  is  why  no  general  computations  of  the 
utility  of  these  systems  are  available  and  why  it  is  difficult 
to  compare  them  with  other  MASK  systems. 

Shagena  and  K varda  (64)  of  Bendix  Corporation  designed 
an  interesting  encoder,  performing  an  operation  which  they 
call  “  varilevel  coding  It  is  also  a  binary  to  multilevel 
translator  with  equal  input  and  output  rate,  which  operates 
under  the  restriction  that  the  q-ary  output  stays  at  the  same 
level  if  the  binary  input  is  a  zero,  and  changes  for  one  step 
up  if  the  binary  input  is  a  one.  Naturally,  after  a  certain 
number  of  ones,  the  translator  output  will  have  reached 
the  highest  level.  The  translator  then  reverses  the  stepping 
direction  and  reduces  the  level  for  one  step  whenever  a 
binary  one  appears  at  the  input.  This  continues  until  the 
lowest  level  is  reached,  when  the  stepping  direction  is  again 


reversed.  The  result  is  a  stepped  zig-zag  waveform,  provided 
all  q-ary  output  samples  are  held  until  the  next  sample  is 
taken.  This  "  staircase  ”  waveform  reaches  the  highest 
slope  when  the  input  consistently  delivers  ones.  The  system 
can  claim  that  it  will  achieve  a  concentration  of  the  waveform 
energy  in  the  low  frequency  part  of  a  low-pass  channel, 
but  it  cannot  avoid  a  de  component  when  the  input  con¬ 
sistently  delivers  zeros. 

Gerrisch  (65)  of  the  Bell  Telephone  Laboratories  compared 
the  low-pass  systems  discussed  above  under  simulated 
practical  conditions.  He  used  the  attenuation  and  phase 
characteristics  of  a  telephone  line  and  assumed  that  a  low- 
pass  signal  would  be  transmitted  in  coherent  amplitude 
modulation  over  this  channel.  He  added  noise,  simulated 
the  various  multilevel  signals,  and  observed  the  eye  pattern  at 
the  receiver  after  the  signal  had  been  contaminated  by  noise 
and  distortions.  He  measured  the  loss  in  signal  margin 
above  the  noise,  as  observed  by  the  narrowing  of  the  eyes, 
and  plotted  this  signal  voltage  degradation  as  a  function 
of  the  transmission  rate.  We  converted  his  data  to  the 
magnitudes  of  the  utility  chart  and  plotted  the  results 
in  fig.  8a.  Curve  8,  a  two-level  (binary)  system,  serves  as  a 
standard  of  reference.  We  started  it  arbitrarily  at  u  =  —  o 
=  —10  db.  The  effective  bandwidth  of  the  low-pass 

channel  described  above  was  close  to  1000  Hz.  This  point 
(u  -  — 10  db  ;  D  =  0  db)  corresponds  to  1000  bits  per  second 
(bps).  One  can  see  that  an  increase  of  about  5  rib  in  SNR 
is  needed  to  operate  this  system  at  D  =  3  db  or  2000  bps 
(Nyquist,  rate  for  this  channel),  keeping  it  at  the  same  error 
ratio  (about  10 ~3).  Proceeding  beyond  the  Nyquist  rate 
causes  such  heavy  distortions  that  the  line  drops  rapidly. 
Curve  !)  is  the  duobinary  system  that  is  clearly  superior 
to  all  the  other  systems.  It  lias  its  maximum  utility  at  the 
Nyquist  rate  where  it  reaches  the  same  utility  value  as  the 
binary  system  at  much  lower  rates.  Naturally,  far  below 
the  Nyquist  rate,  the  binary  system  is  superior.  This  is  the 
power  penalty  one  pays  for  the  multilevel  mode  of  operation. 
The  utility  curve  indicates  that  the  payoff  for  this  penalty 
is  a  fair  return  in  increased  transmission  rate. 

The  comparison  with  the  other  two  multilevel  systems 
shown  in  fig.  8a  is  not  so  favourable,  although  at  highest 
rates  all  systems  are  better  than  the  binary  system.  Curve  1 0 
represents  a  four-level  MASK  system  without  any  coding 
for  wave-shaping.  It  clearly  shows  a  maximum  at  the 
Nyquist  rate.  Beyond  this  rate,  intersymbol  interference 
takes  its  toll  and  the  curve  falls  more  rapidly  than  the  duo¬ 
binary  system.  The  varilevel  coding  system  (curve  11) 
has  the  lowest  utility.  It  has  its  own  maximum  utility 
at  about  70  per  cent  of  the  Nyquist  rate,  but  it  stays  several 
decibels  below  the  other  systems. 

Finally,  we  refer  to  a  special  PCM  system  published  by 
Brogle  (60)  of  Radio  Corporation  of  America.  Ho  arrives 
at  a  ternary  system  by  arithmetically  adding  two  binary 
channels  that  are  assumed  to  operate  synchronously.  If 
both  channels  have  a  Mark,  his  adder  produces  a  positive 
output  level  ;  if  both  channels  have  a  Space,  the  udder 
produces  a  negative  output  level  of  the  same  absolute  value. 
If  one  channel  has  a  Mark  and  the  other  a  Space,  the  adder 
stays  at  zero  level.  To  resolve  the  ambiguity  of  this  last 
case,  Brogle  shifts  the  channels  against  each  other  in  phase 
for  one-half  elementary  length.  The  resulting  biternarv 
output  wave  is  said  to  transmit  a  maximum  rate  of  informa¬ 
tion  with  a  minimum  number  of  pulse  amplitudes. 

In  summarizing,  it  may  be  stressed  that  a  number  of 
ingenuous  ideas  have  been  published  on  the  use  of  MASK 
methods  to  match  binary  information  to  practical  channels, 
and  it  can  be  anticipated  that  some  of  these  methods  will 
gain  importance  when  used  in  combination  with  suitable 
modulation,  coding,  or  multiplexing  systems. 

F5.  Special  Voiceband  Systems  (SSL  and  I  SB) 

Vestigial  sideband  modulation  (VSB)  is  an  amplitude 
modulation  system  that  completely  transmits  one  sideband 
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except  for  the  lowest  frequencies,  which  are  linearly  tapered 
off  to  50  percent  of  their  normal  value  at  the  carrier  frequency, 
while  the  other  sideband  is  completely  suppressed  except 
for  the  lowest  frequency  components,  which  are  raised 
linearly  up  to  50  percent  at  the  carrier  frequency.  This 
treatment  secures  transmission  of  the  lowest  frequencies  in  a 
quasi-double-sidoband  mode  and  it  also  ensures  the  trans¬ 
mission  of  a  dc  component.  This  is  very  important  for  tho 
undistorted  detection  of  pidsive  waveforms  (Murakami  and 
Sonnenfeldt  58). 

Tho  Sebit  25  of  Rixon  Electronics  Inc.  is  most  likely 
the  first  commercial  VSB  data  terminal  (Holland  and  My  rick 
50a,  b).  It  was  designed  in  the  years  before  1959,  was 
transistorized,  and  equipped  with  plug-in  modules  to  adapt 
it  to  various  operational  situations.  It  was  designed  for 
2500-bits-per-second  transmission  rates  and  used  a  carrier 
at  2500  Hz.  It  provides  manual  adjustments  to  compensate 
independently  for  amplitude  and  phase  (or  time  delay) 
distortions  which  the  signals  may  suffer  when  running  over 
typical  telephone  circuits.  A  fast-acting  automatic  gain 
control  (AGC)  system  was  employed  to  accommodate  varia¬ 
tions  of  more  than  50  db  in  the  level  of  the  receiver  input 
signals.  A  rather  complex  timing  system  was  needed  to 
ensure  accurate  resampling  of  the  mutilated  waveforms 
extracted  by  the  VSB  receiver. 

Extensive  further  development  work  in  the  last  eight  years 
added  many  improvements  to  the  terminal  publicized  in 
1959.  Groff  and  Powers  (61)  of  ACF  Industries  Inc.  described 
the  design  and  operation  of  a  “  synchronous  4800  bps 
data  terminal  that  utilizes  quaternary,  AM,  supprossed- 
earrier  vestigial  sideband  transmission  ”.  This  equipment 
employs  a  two-step  encoder  :  in  the  first  step,  the  binary 
input  signals  are  changed  into  a  form  in  which  the  information 
is  carric  l  in  the  transitions  from  one  level  to  the  other,  rather 
than  in  ihe  levels  themselves  ;  in  the  second  step,  the  signals 
resulting  from  the  first  step  are  converted  into  quaternary 
samples.  This  binary-to-quarternary  conversion  may  be 
bypassed  when  operating  at  2400  bits  per  second.  The 
quaternary  samples  are  preshaped  by  a  shaping  filter  and 
then  modulated  in  a  balanced  modulator  to  a  2400-Hz 
carrier  which  is  synchronized  to  the  input  data  rate  (via  2  :  1 
or  1  :  1  divider  respectively).  The  output  of  the  balanced 
modulator  is  run  through  a  vestigial  sideband  filter.  The 
receiver  uses  an  interesting  arrangement  with  a  special 
inter-symbol-interferonce  corrector  (ISIC)  consisting  of  16 
stages  of  delay  eloments,  the  outputs  of  which  are  summed 
in  a  special  adjustable  network  (Gibson  Ola,  b). 

The  development  of  a  vestigial  sideband  phase-reversal 
data  transmission  system  is  reported  by  International  Busi¬ 
ness  Machines  Corporation  (McAuliffe  62,  Critchlow  64). 
This  system  employs  an  improved  method  of  carrier  recovery 
for  VSB  systems  which  frees  the  designer  from  the  constraint 
that  the  bit  rate  and  carrier  frequency  have  to  be  syn¬ 
chronized.  A  terminal  of  this  system  can,  for  example, 
accept  teletypewriter  signals  in  place  of  synchronously 
clocked  binary  data.  The  carrier  recovery  system  uses  a 
complementary  VSB  filter  that  has,  in  the  vicinity  of  the 
carrier  frequency,  the  opposite  frequency  rosponse  of  the 
original  VSB  filter  used  in  tho  transmitter.  Together  the 
two  filters  form  a  narrow  bandpass  characteristic  of  half¬ 
cosine  shape  with  maximum  response  at  the  carrier  frequency. 
The  r  ;st  of  the  carrier  recovery  circuitry  is  identical  to  the 
usual  double  sideband  (DSB)  carrier  recovery  systems  with 
full-wave  rectifier  and  an  optional  phase-locked  oscillator. 
The  1 BM  team  performed  functional  tests  with  a  four-level 
mode  of  this  VSB  data  system.  These  tests  were  conducted 
in  the  laboratory  over  the  carrier  channel,  the  characteristics 
of  which  are  not  specified  in  detail.  Assuming  a  bandwidth 
of  3390  Hz  for  this  carrier  channel,  one  can  plot  the  results 
in  the  utility  chart  (fig.  8a).  Curve  12A  corresponds  to  the 
utility  of  the  system  if  the  data  carrier  is  patched  through 
between  transmitter  and  receiver.  Curve  12B  applies  when 


the  carrier  is  recovered  in  the  receiver  from  the  noisy  signal. 
Noise  from  a  random  noise  generator  was  added  at  the  input 
to  the  carrier  channel.  Both  curves  aro  for  an  error  ratio 
of  1 0“5,  the  only  value  given  by  Critchlow.  Field  tests 
were  performed  with  4000  bps  over  a  long  distance  line  and 
with  8000  bps  over  a  signal  carrier  link.  Eye  diagrams 
are  supplied  as  the  results  of  these  tests. 

The  VSB  system  described  above  was  subsequently  used  to 
test  the  technical  feasibility  of  automatic  distortion  compensa¬ 
tion  (Schreiner  et  al  65).  Tho  results  of  these  tests  were 
encouraging,  but  the  authors  kept  the  possibility  open  that 
“  systematic  approaches  to  automatic  time  domain  equaliza¬ 
tion  other  than  thoso  discussed  [by  them]  may  lead  to  even 
better  results  ”. 

Another  team  of  IBM  researchers  from  France  described 
an  interesting  improvement  which  amounts  to  a  combination 
of  the  wave-shaping  binary  to  q-ary  coding  procedures 
discussed  in  subsoction  F4  and  a  SSB  modulation  method 
(Melas  and  Gorog  64).  This  system  has  an  encoder  that 
uses  four  binary  elements  (a,  ,  a,  ,  a3  ,  a,)  and  converts 
them  into  four  quaternary  elements  (8,  ,  Pt  ,  p3 ,  Pi),  each 
selected  from  an  alphabet  (A,  B,  C,  D)  of  four  bipolar  ampli¬ 
tude  levels  (for  example  :  +  3,  +1,  —I,  —3).  The  encoding 
rules  are  the  following  : 

Pi  =  “i  («s  +  -) 

Pi  =  «»  (*i  +  2) 

(35) 

Pa  —  ®i  (“a  +  2)  =  —  pt 

Pi  =  -  “s  («4  +  2)  =  -  p2 

Naturally,  as  tho  rate  of  the  binary  input  is  tile  same  as  the 
rate  of  the  quaternary  output,  the  encoder  introduces 
redundancy.  This  redundancy  is  used  to  eliminate  the  do 

4 

component  (  X  Pi  =  0)  and  to  shape  the  spectrum  so  that  the 

i 

highest  essential  frequency  component  is  one  half  of  the 
digital  rate.  These  quaternary'  output  digits  of  the  encoder 
are  held  for  one  elementary  duration  each  so  that  they  form  a 
sequence  of  rectangular  steps.  This  quaternary  step 
waveform  is  then  passed  through  a  low-pass  filter  with  about 
2500-Hz  cutoff  before  it  is  submitted  to  the  SSB  modulator, 
where  it  is  modulated  to  a  carrier  of  2800  Hz.  Tho  lower 
sideband  is  separated  by  a  low-pass  filter  of  3000-Hz  cutoff 
frequency  and  the  output  of  this  filter  is  used  as  the  trans¬ 
mission  signal.  Tho  result  of  this  combined  encoding  and 
modulation  process  is  a  nonbinary  transmission  signal  with  a 
spectrum  well  matched  to  a  telephone  channel  and  with  the 
double  advantage  of  having  no  dc  component  and  no  informa¬ 
tion  dependent  component  at  the  carrier  frequency  of  2800  Hz. 
A  rest  carrier  is  purposely'  transmitted  and  can  easily'  be 
used  in  the  receiver  for  synchronous  demodulation.  There 
is  no  need  to  lock  the  data  rate  to  the  carrier  frequency, 
but  there  is  also  no  ill  effect  if  this  is  done  for  the  sake  of 
simpler  synchronization.  The  receiver  decoder  uses  a 
resampler  which  samples  only  the  /3,  and  p2  positions,  but 
neglects  to  sample  the  p 3  and  p,  positions.  One  of  tho  latter 
ones  shows  intersymbol  interference  after  the  shaping, 
modulation,  and  demodulation  processes.  Eye  diagrams 
demonstrate  tho  small  amount  of  intersymbol  interference 
at  the  correct  sampling  instance  when  operating  the  system 
with  4800  bps.  The  authors  claim  that,  at  this  rate,  tests 
over  4B-typo  telephone  lines  yielded  10-s  error  ratio  for 
20-db  power  contrast  (SNR).  This  places  the  system  at 
operating  point  13  (D  =  2.67  db  ;  u  =  —17.3  db)  in  fig.  8a. 

McAuliffe  (64)  of  North  American  Aviation  Inc.  describes 
a  simplification  of  the  VSB  transmission  system  discussed 
by  Critchlow  et  al  (64).  This  simplification  concerns  the 
VSB  filter  at  the  transmitter,  which  now  can  be  designed 
without  delay  equalization.  This  has  become  possible 
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because  of  the  application  of  a  novel  method  of  automatic 
data  equalization,  which  equalizes  the  data  transitions 
dynamically.  The  static  amplitude  and  phase  characteristics 
are  ignored  ;  instead,  attention  is  focused  on  the  dynamic 
time  response  of  the  channel  to  data  transitions.  All 
modem  (modulator-demodulator)  filters  may  be  considered 
as  part  of  the  channel  and  the  complete  complex  is  equalized 
at  once.  In  this  technique  the  equalization  is  actually 
matched  to  the  specific  signal  used,  which  is  why  it  is  termed 
“  data  equalization  The  goal  of  the  technique,  which 
operates  on  a  digital  basis,  is  shaping  the  input-digits, 
together  with  interlaced  dummy  digits,  in  such  form  that  the 
output  at  the  receiver  gives  the  largest  signal  change  at  the 
decision  moment  for  any  input  data  change.  The  technique 
is  described  by  the  author  with  reference  to  digital  pre- 
distortion  in  the  transmitter,  but  he  later  extends  his  discus¬ 
sion  to  post-detection  equalization.  The  terminal  incorporat¬ 
ing  this  improvement  is  called  Adaptively  Data  Equalized 
high  speed  Modem  (ADEM).  It  is  claimed  that  up  to  5400 
bps  can  be  transmitted  when  combining  ADEM  with  a  VSB 
modulation  mode. 

The  most  refined  development  of  VSB  data  transmission 
equipment  is  in  progress  in  the  Bell  Telephone  Laboratories. 
Starting  with  a  1650-bps  system  (Brand  and  Carter  62), 
released  around  1961,  staff  members  of  this  organization 
described  improved  VSB  equipment  with  the  capability  to 
transmit  asynchronously  data  rates  at  2000  bps  over  the 
switched  telephone  network  and  at  3000  bps  over  equalized 
voiceband  circuits  (Becker  et  al  62).  A  later  release  from 
the  same  laboratories  describes  an  experimental  multilevel 
VSB  terminal  for  2400,  4800,  7200  and  9600  bits  per  second 
(Becker  65)  operating  with  2,  4,  8  and  16  levels,  respectively. 
Becker  concludes  that-  the  following  condit  ions  must  be  met 
when  designing  an  optimum  data  transmission  system  : 

(1)  Choose  the  modulation  system  with  the  best  bandwidth 
utilization. 

(2)  Equalize  delay  and  the  amplitude  characteristic  of  the 
channel  automatically  so  as  to  exploit  its  full  bandwidth. 

(3)  Couple  the  system  with  error  correction  so  as  to  meet 
the  accuracy  requirement  of  the  system  while  also 
maximizing  the  data  rate. 

It  is  interesting  to  notice  that  these  three  conditions  are 
in  line  with  the  cardinal  principles  No.  4,  No.  2,  and  No.  6, 
part  of  the  ten  cardinal  principles  to  be  followed  for  the 
optimization  of  an  integrated  data  transmission  system 
(Filipowskv  59). 

To  meet  the  three  conditions  postulated  above,  Becker  (65) 
explains  that  the  best  design  involves  a  multilevel  VSB 
modulation  mode,  described  in  his  paper  ;  an  automatic 
equalization  system  described  in  a  paper  by  Lucky  (65)  ; 
and  an  error  control  system  described  in  a  paper  by  Burton 
and  Weldon  (65b). 

The  combined  effect  of  the  application  of  these  three 
principles  is  an  integrated  data  system  that  can  operate 
up  to  4.1  db  bit  density  at  —23.9  db  utility  and  up  to  6.3  db 
bit  density  at  —32.2  db  utility.  This  operating  line  is 
placed  as  line  14  into  fig.  8a.  Using  fig.  8  from  Becker  (65) 
and  using  the  average  line  characteristics  for  background 
noise  from  Alexander  et  al  (60),  the  systems  performance 
has  been  calculated  in  the  units  for  our  utility  chart,  assuming 
an  average  minimum  signal  strength  at  the  receiver  input 
of  —32  dbm.  This  gives  a  power  contrast  of  28  db  for  long¬ 
distance  lines  and  38.5  db  for  short-haul  lines.  The  effective 
bandwidth  is  assumed  at  2000  Hz  from  fig.  2  of  Becker  (65). 
Pidsive  noise  is  not  directly  considered  in  the  determination 
of  the  power  contrast  but  indirectly  by  plotting  line  14  for  an 
error  ratio  of  10-5,  keeping  in  mind  that  the  system  operates 
with  a  complex  error-correction  system.  It  is  interesting 
to  note  that  curve  14  falls  right  into  the  theoretical  range 
Predicted  by  Arthurs  and  Dym  (62)  for  a  coherent  MASK 
tystem.  Notice,  however,  that  the  theoretical  model  is  for 


monopolar  multilevel  signals  in  DSB  ;  it  is  plotted  for  an 
error  ratio  of  10- 3  in  white  noise  ;  and  it  does  not  assume 
eiTor  correction.  The  practical  system  of  the  Bell  Telephone 
Laboratories  team,  as  plotted  for  10-5  error  ratio,  operates 
over  a  variety  of  circuits  under  natural  noise  conditions, 
employs  VSB,  and  uses  an  error-correction  system.  Despite 
theso  differences  it  is  significant  that  the  results  of  the  theoreti¬ 
cal  analysis  and  of  the  operational  tests  of  a  MA.SK  system 
fall  into  the  same  small  area  of  our  utility  chart.  It  is  surely 
an  indication  of  the  high  engineering  standard  of  this  terminal, 
which  operates  at  twice  the  Nyquist  rate. 

This  remarkable  achievement  of  the  Bell  Telephone  Labora¬ 
tories  terminal  is  due  to  a  new  signal  format  with  two  pilot 
carriers  at  600  Hz  and  3000  Hz  (Becker  et  al  66),  a  very 
sophisticated  automatic  equalization  system  (Lucky  65a, 
b,  Becker  et  al  65),  and  an  efficient  error  control  system 
(Burton  and  Weldon  65a,  b).  The  carrier  and  timing 
recovery  subsystem  is  a  design  with  a  previously  unbeliet'ablo 
precision  of  operation.  Figure  9a  shows  the  eye  pattern 
of  this  terminal  in  the  binary  mode  (2400  bps)  and  in  the 
octonary  mode  (7200  bps),  along  with  a  raised  cosine  pulse 
waveform.  Figure  9b  demonstrates  the  improvement  due 
to  equalization  with  the  automatic  equalizer  described  by 
Lucky  (65a,  b). 

F6.  Carrier- Band  Systems 

While  VSB  is  an  excellent  modulation  method  for  matching 
digital  signals  to  the  special  characteristics  of  a  voice-band 
channel,  other  methods  may  be  preferable  for  general  types 
of  carrier-band  channels  like  those  used  in  radio  links. 

The  advantages  of  double  sideband  transmission,  primarily 
in  the  suppressed  carrier  mode,  have  been  stressed  in  many 
publications  (Costas  59,  Gibby  60,  Tsikin  62).  The  previously 
mentioned  varilevel  coding  method  of  Bendix  Corporation 
(Shagena  anil  Kvarda  64)  is  used  in  connection  with  a  DSB 
modulation  method,  placing  the  carrier  at  1800  Hz  into  the 
centre  of  a  voice-band.  A  balanced  modulator  and  a  syn¬ 
chronous  demodulator  are  used  to  handle  the  suppressed 
carrier  signals. 

The  Philips  Research  Laboratories  worked  on  the  develop¬ 
ment  of  “co-modulation”,  a  new  method  of  high-speed  data 
transmission  (DeJager  and  Van  Gerwen  62).  The  name 
stands  for  complementary  orthogonal  modulation.  The 
primary  object  is  to  operate  at  higher  bit  densities  than 
standard  systems.  The  essential  features  of  the  system 
involve  : 

(1)  Suppression  of  the  lower  frequencies  in  the  base-band 
signals  before  modulation  with  the  help  of  a  high-pass 
filter. 

(2)  Multiplexing  of  two  binary  channels,  by  using  two 
carrier  voltages  differing  90  degrees  in  phase. 

(3)  Restoring  in  the  receiver  the  frequency  components, 
originally  suppressed  in  the  transmitter,  with  the  help 
of  a  feedback  network. 

Feature  1  enables  the  system  to  insert  a  pilot  signal  at  the 
carrier  frequency  for  phase  reference  and  for  coherent 
detection.  Feature  2  causes  the  transmission  signal  to  look 
like  a  quaternary  phase-modulated  signal  when  both  channels 
are  keyed  synchronously. 

An  experimental  system  operated  with  a  rate  of  4000  bps 
over  a  normal  carrier-telephone  channel  in  a  band  from  700  Hz 
to  3100  Hz,  yielding  a  bit  density  of  1.67.  Phase  equaliza¬ 
tion  was  necessary  over  certain  kinds  of  lines.  O’Neill 
and  Saltzberg  (66)  described  an  automatic  equalizer  for  this 
class  of  coherent  quadrature-carrier  data  transmission 
systems. 

F7.  Other  MASK  Developments 

In  addition  to  the  special  development  efforts  reported 
in  the  above  subsections.  Bason  (66).  in  Sydney,  designed  a 
transistorized  experimental  model  of  a  multilevel  system 
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(o)  (f>) 

Figure  9. — («)  Unequalizett  operation. 

(b)  Equalized  operation. 

( Courtesy  of  II.  IF.  Lucky,  Bell  Tel.  Lab.  and  IEEE,  Mem  Vork.) 
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for  the  simulation  of  various  signal,  channel,  anil  noise 
combinations.  This  experimental  setup  permits  the  setting 
of  linear  or  nonlinear  threshold  adjustments  and  provides 
for  (the  simulation  of  manv  message  types  with  rates  up  to 
4000  bps. 

Litchuum  (64)  of  ITT  Communications  Systems  Inc. 
examined  the  technical  prospects  for  handling  large  volumes 
of  digital  traffic  over  submarine  cables.  He  develops 
arguments  for  an  all-digital  cable  and  describes  in  detail  an 
economical  approach  using  existing  repeater  and  cable 
technology  for  the  transmission  of  multilevel  digital  data 
at  rates  in  excess  of  1 .5  megabits  per  second. 

JLebow  et  al  (63)  of  Lincoln  Laboratory,  Massachusetts 
Institute  of  Technology,  report  another  ultra-high  rate 
experimental  data  system  of  similar  integrated  character  as 
tile  Bell  Telephone  Laboratories  system  described  previously. 
It  is  a  truly  adaptive  variable  rate  system  and  it  combines 
the  following  features  : 

i  (1)  The  encoder  is  a  self-regulating  error-correcting  coder- 
encoder  (SECO)  (Wozencraft  and  Reiffen  61,  Perry 
and  Wozencraft  62),  which  can  operate  with  redun¬ 
dancies  of  eight  different  values,  of  which  three  were 
used  in  the  tests  reported  here  :  3/5,  2/5,  1/5.  It 
is  a  system  that  takes  a  long  sequence  of  input  bits 
(60  in  this  case)  and  inserts  an  average  of  (1/r  —  1) 
parity  check  bits  for  each  data  bit,  with  r  assuming 
the  above  values  of  redundancy.  From  the  received 
data  bits,  the  decoder  generates  the  same  sequences 
that  the  coder  would  generate  hail  it  actually  received 
the  decoded  message  as  input.  For  each  newly 
received  bit,  the  decoder  attempts  to  find  that  hypo¬ 
thetical  input  message  that  agrees  sufficiently  well  with 
the  one  resulting  from  the  decoded  noisy  message. 
This  process  is  updated  on  a  sequential  basis  so  that  a 
duster  of  errors  may  be  corrected  if  it  is  followed  by  a 
sufficiently  long  string  of  correct  bits  before  the  60 
bits  are  reached  that  constitute  the  duration  involved 
in  the  process. 


(2)  The  encoder  also  contains  a  translator  to  a  q  =  32 

multilevel  alphabet.  Assume  that  the  input  of  the 
Lincoln  Laboratory  system  operates  at  the  highest 
rate  of  9000  bps  and  that  r  3/5  is  selected.  This 
means  that,  in  the  translator,  the  encoder  handles 
9000  -f  5400  =  14,400  bps.  The  output  of  the  trans¬ 
lator  will  submit  14,400/log.  32  =  2880  multilevel 

signals  to  the  modulator. 

(3)  First,  the  modulator  has  a  low-pass  pulse-shaping 
network,  which  generates  with  the  help  of  5  microsecond 
pulses  and  a  tapped  delay  line  and  output  low-pass 
waveform  with  precise  nulls  at  intervals  corresponding 
to  the  digital  rate  (2880  Hz  in  this  case).  This  signal- 
synthesis  network  produces  a  low-pass  spectrum  which 
best  matches  the  telephone  lme  characteristics  when 
translated  into  a  VSB  signal  with  the  help  of  the  actual 
modulator,  using  a  carrier  at  2500  Hz. 

(4)  A  pilot  tone  at  3000  Hz,  together  with  the  carrier, 
ensures  phase-correct  synchronous  demodulation  even 
in  the  presence  of  a  phase  crawl  over  the  carrier  tele¬ 
phone  system. 

(5)  The  receiver  demodulator  has  a  very  sensitive  carrier- 
extraction  system  and  operates  as  a  synchronous 
demodulator. 

(6)  The  decoder  operates  as  described  under  1  and  measures 
its  own  ability  to  decode  rapidly  and  efficiently.  This 
can  be  done  by  noticing  the  decoding  distances  and 
the  buffer  delay  during  operation.  When  these 
values  increase  too  fast  the  receiver  will  request  over 
a  return  channel  a  slowing  down  of  the  rate  and/or 
an  increase  of  redundancy.  When  the  opposite  is  the 
case,  the  receiver  will  request  an  acceleration  of  the 
operation. 

This  SECO- VSB  system  operates  practically  error-free 
up  to  its  breakdown  point.  The  authors  report  that  an 
experimental  system  was  operated  in  1963  over  an  800-mile 
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loop  of  a  type  K  carrier  system  with  t  hroughput  rates  from 
0000  to  9000  bps.  Using  average  values  of  the  power 
eont  rast  for  such  circuits  places  this  MASK  system  at  a  higher 
utility  and  bit  density  than  the  Boll  Telephone  Laboratories 
system  of  similar  structure.  Notice,  however,  that  the 
SECO-VSB  system  needs  a  return  channel  and  that  the  tests 
reported  by  Lebow  et  al  (03)  were  conducted  with  a  noise- 
free  return  channel. 

Concluding  the  section  on  MASK  systems  it  may  be  stated 
that  a  large  variety  of  developments  are  in  progress  and  that 
this  class  of  systems  seems  to  be  well  qualified  to  fill  the  upper 
part  of  the  right  side  of  the  utility  chart.  The  MASK-VSB 
combination  seems  to  emerge  as  the  most  efficient  solution 
for  this  class  of  nonbinary  systems. 

G.  Multifrequency  Shift  Keying  (MFSK)  Systems 

Gl.  Fundamentals  of  FSIi  and  MFSK  Systems 

The  same  reasons  that  made  frequency  modulation  (FM) 
so  popular  are  also  the  reasons  for  the  wide  application  of 
its  digital  counterpart,  usually  known  under  the  name 
frequency  shift  keying  (FSK).  It  has  long  been  known  as  a 
carrier  telegraph  method  ;  radio  telegraphy  used  this  mode 
of  operation  for  several  decades,  benefiting  from  the  low 
sensit  ivity  to  fading  and  noise  which  characterizes  particularly 
the  wide-band  FSK  mode  (Jones  and  Pfleger  46,  Davey 
and  Matte  48,  H.  O.  Peterson  et  al  46). 

Conventional  radio  telegraphy  operates  with  two  states  : 
Mark  and  Space.  In  the  FSK  mode  each  state  corresponds 
to  a  special  constant  frequency.  The  receiver  has  the  a 
priori  knowledge  of  these  two  frequencies  and  its  task  is  to 
extract  the  signals  from  noise  and  to  decide  which  of  the 
two  frequencies  has  most  likely  been  transmitted  in  a  given 
interval.  Naturally,  to  avoid  confusion,  these  two  fre¬ 
quencies  should  be  so  far  apart,  that  filtei's  t  uned  to  one  would 
not  be  affected  by  the  signals  shifted  to  the  other  frequency. 
On  the  other  hand,  one  would  like  to  keep  them  close  enough 
together  that  the  combined  spectrum  of  Mark  and  Space 
signals  would  have  t  he  minimum  band  occupancy  consistent 
with  the  desired  digital  transmission  rate.  In  terms  of  the 
definitions  in  Part  I,  this  means  that  the  signal  base  f)  would 
have  the  minimum  value  for  operation  with  acceptable 
signal  distortions,  for  binary  FSK  fi  is  usually  between  1.2 
and  1.8  (Feldman  and  Faraone  (61).  Multifrequency  shift 
keying  systems  are  basically  an  extension  of  the  binary 
FSK  systems.  More  than  two  frequency  states  are  used  at 


the  transmitter  and  the  receiver  has  the  more  difficult  task 
of  making  a  nonbinary  decision.  Figure  4  showed  the 
principal  elements  of  an  MFSK  system  using  eight  frequencies 
to  transmit  three  bits  of  information  with  each  pulsive 
signal.  Many  combinations  of  transmitter  methods  and 
receiver  methods  are  conceivable  for  FSK  systems. 

Figure  10  shows  on  the  left  side  three  different  arrange¬ 
ments  of  the  transmitting  side  of  an  FSK  terminal.  On  t  he 
top,  bottom,  and  right  side  are  six  arrangements  for  the 
receiving  side.  Principally,  each  transmitting  block  diagram 
could  be  combined  with  any  one  of  the  six  receiving  block 
diagrams  to  form  a  workable  system.  Yet  certain  combina¬ 
tions  arc  more  efficient  than  others.  Those  preferred  in 
practice  are  indicated  by  shaded  areas  in  the  centre  of 
fig.  10. 

The  combination  T,-R,  is  possibly  the  lirst  FSK  system 
that  has  been  brought  into  operation.  It  simply  performs 
exactly  what  has  been  defined  above  as  FSK.  There  are 
two  oscillators  operating  on  F,  and  F,  ,  but  two  mutually 
exclusive  analog  gates  (or  one  relay)  permit  only  one  of  the 
two  frequencies  to  reach  the  transmission  channel.  The 
binary  source  controls  these  gates.  The  receiver  R,  has  t  wo 
bandpass  filters  centred  on  F,  and  Fs  to  separate  Mark 
from  Space  signals.  If  only  binary  signals  are  to  be  received, 
the  receiver  will  preferably  operate  in  a  clocked  mode  with  a 
synchronizing  circuit  triggering  the  decider  at  the  end  of 
each  digital  interval  (Glenn  66).  This  combination  T,-R, 
is  sometimes  known  as  noncoherent  FSK,  but  we  prefer 
the  term  switched  FSK,  reserving  the  term  noncoherent 
for  any  FSK  method  applying  the  receiver  arrangement 
R,  or  R4  but  not  necessarily  using  the  transmitter  arrange¬ 
ment  T,  .  A  system  may  bo  coherent  on  both  ends,  on  either 
end,  or  nowhere.  Little  attention  is  given  in  the  literature 
to  this  ambiguity  of  the  term  coherent,  though  Bennett  and 
Rice  (63)  discriminate  clearly  between  waveforms  with 
discontinuous  phase  (case  T, )  and  waveforms  with  continuous 
phase  (case  Ta),  while  many  authors  discriminate  between 
coherent  FSK  and  noncoherent  FSK,  assuming  continuous 
phase  on  the  transmitting  side  but  coherent,  or  noncoherent 
detect  ion  on  the  receiving  side.  The  output  of  the  arrange¬ 
ment  T,  can  be  made  continuous  when  the  difference  between 
the  two  frequencies  is  made  an  exact  integer  multiple  of  the 
binary  input  rate,  assuming  perfectly  constant  frequencies 
and  instantaneous  switching  action. 

The  combination  'IVR.  can  lead  to  a  coherent  FSK  system 
if  the  local  oscillators  in  the  receiver  can  be  synchronized 


Figure  10. — Circuit  arrangements  of  binary  FSK  systems. 
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and  correctly  phase  locked  to  the  respective  oscillators  in 
the  transmitter. 

Any  of  the  combinations  mentioned  thus  far  can  be  used 
in  connection  with  the  optional  carrier  modulators.  Such 
systems  frequently  carry  the  designation  FSK-XM,  where  X 
stands  for  the  kind  of  carrier  modulation  applied.  The  most 
frequently  used  system  is  FSK-AM,  primarily  FSK-SSB 
(Lender  65b,  Glenn  59,  60).  FSK-FM  is  usually  discussed 
as  a  pulse  code  modulation  system  and  designated  PCM-FM- 
FM,  where  the  first  PCM-FM  is  closely  related  to  the  third 
preferred  FSIv  combination  to  be  discussed  below  in  connec¬ 
tion  with  arrangement  T3  . 

An  important  combination  is  T,-R3  .  This  is  generally 
called  the  orthogonal  FSK  system,  although  the  arrangement 
T,  can  also  produce  a  pair  of  orthogonal  waveforms  when 
enforcing  certain  ratios  between  the  bit  rate,  F,  ,  and  F,  . 
Some  authors  call  any  system  orthogonal  which  is  free  of 
cross  talk  between  Mark  and  Space  waveforms,  thus  tying 
the  orthogonality  to  the  receiver  extractor  characteristics. 
The  advantage  of  the  combination  T2-R3  is  that  it  is  theo¬ 
retically  optimum,  and  that  it  offers  the  designer  the  possi¬ 
bility  of  shaping  the  amplitude  of  the  FSIv  waveforms  for 
matching  them  to  a  bandlimited  channel. 

The  remaining  preferred  combinations  are  systems  using 
T3  as  transmitter  terminal  and  either  R,  ,  R5  ,  or  R„  as 
receiver  terminal.  The  block  diagram  for  T3  basically 
shows  an  analog  FM  modulator  that  receives  a  low-pass 
waveform  as  information  carrier.  The  resulting  signal  is 
usually  designated  as  binary  FM.  This  arrangement  ensures 
a  waveform  without  phase  discontinuities.  A  dc  component 
can  be  removed  by  introducing  into  the  base-band  signal  a 
rectangular  carrier  of  the  bit  rate,  frequently  called  the  clock. 

The  three  preferred  receiver  arrangements  R4  ,  Rs  ,  R„ 
show  significant  differences.  However,  they  all  have  in 
common  the  improvement  threshold.  In  the  terminology 
of  Part  I  of  this  paper  we  may  say  that  the  binary  FM  systems 
call  for  a  nonlinear  signal  processor  (Part  I,  section  C3). 
R4  is  the  straightforward  FM  receiver  arrangement  and, 
indeed,  it  is  frequently  applied  in  binary  FM  (Lender  65b, 
Glenn  60,  Bennett  and  Salz  63b,  Zabronsky  61,  Meyerhoff 
and  Mazer  61,  Klapper  66).  All  these  publications  discuss 
the  various  aspects  of  the  threshold  problem.  Schilling  et 
al  (65)  pointed  out  that  the  nonlinear  extractors  cause 
rather  flamaging  noise  spikes  in  the  output  of  a  limiter- 
discriminator  circuit,  even  if  the  input  is  contaminated  by 
Gaussian  noise.  This  effect  seems  to  be  the  reason  that  the 
error  rates  of  actual  measurements  differed  considerably 
from  the  calculated  values. 


The  receiver  arrangement  R5  uses  a  phase  lock  loop. 
Schilling  et  al  (65)  compares  this  arrangement  with  R4  . 
It  is  shown,  by  using  Schilling’s  equivalent  circuit  (Schilling 
and  Billig  64),  that,  by  proper  adjustment  of  the  trans¬ 
mission  parameters  (frequency  deviation  and  data  rate), 
errors  due  to  spikes  can  be  made  equal  to  zero.  This  result 
was  experimentally  verified  with  a  receiver  arrangement  of 
the  type  R5  . 

The  recoiver  arrangement  R,  finally  employs  the  FM 
demodulator  with  feedback  (FMDFB)  loop,  which  recently 
has  been  much  used  in  space  communications.  Its  analysis 
is  similar  in  complexity  to  the  analysis  of  the  phase  lock 
loop,  but  Schilling  and  Billig  (64)  extended  Rice’s  technique 
(Rice  63)  to  these  circuits.  They  achieved  results  which  are 
valid  above,  at,  and  below  threshold  and  which  are  easily 
obtained  because  of  the  simplicity  of  the  equivalent  circuit. 
Other  references  to  FMDFB  circuits  may  be  found  in  Van 
Trees  (63),  Ruthroff  (62),  Enloe  (62),  and  in  Giger  and 
Chaffee  (63). 

The  MFSK  systems  may  be  subdivided  roughly  into  the 
same  three  classes  as  the  binary  FSIv  systems,  although 
many  hybrid  systems  are  also  conceivable  in  the  MFSK 
case.  This  classification  leads  to  an  interesting  map  of 
“  spheres  of  influence  ”  of  the  various  classes  when  marking 
their  general  operating  areas  in  the  utility  chart  (fig.  11). 
On  the  far  left  wing  (lowest  bit  density)  are  the  large  TB 
(wide-band)  FSK  systems  (section  G4).  The  range  from 
—  4  to  —20  db  bit  density  is  the  domain  of  the  small  TB 
systems  of  subsection  G3.  The  centre  belongs  to  the  binary 
FSK  systems.  The  extreme  right  (i.e.,  the  area  of  high 
bit  densities)  can  be  reached  only  by  MFSK  systems  of  the 
class  designated  as  MASK-FM  (subsection  G5).  They  are 
hybrid  systems  and  operate  with  extremely  small  FM 
deviation  ratio.  Thus,  we  see  that  theory  does  not  concede 
them  an  advantage  over  direct  MASK  systems.  Indeed, 
the  typical  utility  curve  for  MASK-FM  systems  will  lie 
below  the  corresponding  MASK  curve,  notwithstanding 
some  practical  advantages  that  the  MASK-FM  system  may 
offer  in  special  cases. 

Concluding  this  subsection  on  MFSK  fundamentals,  wo 
would  like  to  stress  the  fact  that  fig.  11  shows  the  versatile 
character  of  MFSK  systems  ranging  over  five  to  seven  decades 
of  bit  density  and  offering  reasonable,  though  not  optimum, 
utility.  Notice,  however,  that  fig.  1 1  is  merely  a  schematic 
diagram  in  which  only  the  large  TB  utility  curves  are  taken 
from  actually  published  values  (Glenn  66),  while  all  the  other 
utility  curves  are  typical  representatives  of  their  class 
without  reference  to  any  special  investigation.  Accurate 


Figure  11. — Typical  utility  curves  of  various  MFSK  classes. 
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utility  curves  for  the  other  classes  will  be  given  in  fig.  12 
and  will  be  explained  in  the  following  subsections. 

02.  Ideal  MFSK  Systems 

It  is  again  convenient  to  start  with  the  ideal  binary  FSIv 
systems.  We  showed  in  fig.  1,  line  3,  the  utility  curve  of  the 
ideal  binary  system,  which  has  been  repeated  in  fig.  8a. 
Section  FI,  equation  34,  shows  that  the  ideal  binary  system 
has  a  constant  utility  for  each  value  of  binary  error  ratio. 
For  c/<  =  1 0- 3,  this  value  is  —6.8  db.  The  ideal  binary 
system  requires  a  correlation  coefficient  of  —1,  which  can 
only  be  readied  with  two  waveforms  being  the  negative  of 
each  other.  FSK  waveforms  must  have  a  cross-correlation 
coefficient  larger  than  —1,  with  0  being  a  preferred  value 
in  many  cases  (orthogonal  binary  FSK).  Under  ideal 
conditions  they  should  approach  the  value  of  formula  (33) 
for  A  =  0  ;  i.e.,  their  utility  curve  should  be  governed  by 
the  equation  : 

ob  =  q{ —1  •  (30) 

IVuJ 

This  is  again  a  straight  line  when  plotted  for  constant  e«  . 


contrast.  Under  these  assumptions,  Lawton  arrives  at  the 
following  equation  : 

=  1 .  e-,/'2“l  (37) 

Solving  for  u  gives  : 


or  in  decibels  : 

u"  =  -3.01  -  10  log|Q  [-0.603  -  log,  5fl]  .  (38a) 

For  the  special  case  5b  =  10-3,  one  can  find  log,  10-3  = 
—  6.008  and  u  =  —10.9  db.  Again  it  can  be  seen  that  the 
binary  utility  curve  is  a  straight  line.  It  is  plotted  as  line 
12B  in  fig.  12. 

Turning  from  binary  mathematical  models  to  MFSK 
models,  we  see  in  fig.  12  the  utility  curves  of  two  different 
models,  published  at  about  the  same  time.  From  Arthurs 
and  Dym  (62),  we  derived  curves  6A  and  6B  ;  Akima  (63) 
published,  apparently  independently,  equations  and  charts 
for  a  slightly  different  MFSK  model.  Akima’s  results  arc 
plotted  as  curves  5A  and  5B  in  fig.  12. 


Figure  12. — Utility  curves  of  MFSK  systems. 


Using  our  favourite  value  5b  =  10~3  leads  (with  the  help 
of  fig.  8a)  to  u  =  —9.9  db.  This  result  is  3  db  less  than  the 
ideal  binary  system,  a  fact  which  may  be  verified  when  com¬ 
paring  equations  36  and  34.  To  act  as  a  wide-hand  system 
and  to  maintain  A  =  0  requires  that  D  should  not  be  larger 
than  about  —4  db.  This  ideal  binary  coherent  FSK  system 
is  shown  in  fig.  12  as  curve  12A  along  with  curves  1  (Shan¬ 
non’s  upper  bound)  and  3  (ideal  coherent  binary  matched 
filter  system,  equation  34),  and  with  many  others  that  will 
be  explained  throughout  this  section. 

Lawton  (58)  derived  the  equation  for  the  error  ratio 
of  an  ideal  non-coheront  FSK  system  with  matched  filters 
after  envelope  detectors.  This  corresponds  to  the  receiver 
arrangement  R,  in  fig.  10,  with  the  low-pass  filters  L,  and  L., 
acting  as  matched  filters.  This  ideal  system  also  assumes 
that  the  decider  will  sample  the  filter  outputs  precisely 
at  the  end  of  the  digital  interval  and  that  the  envelope 
detectors  offer  ideal  operation,  also  at  a  very  small  power 


Arthurs  and  Dym  give  an  equation  (number  64  in  their 
paper)  for  the  error  ratio  of  a  coherent  MFSK  system  operat¬ 
ing  with  matched  filters  or  correlators.  They  derive  the 
character  error  ratio  that  we  had  to  convert  to  the  bit  error 
ratio  to  get  the  utility  curve.  We  discussed  this  problem 
in  section  Cl.  For  the  present  purpose  we  took  the  lower 
limit  of  equation  2  and  converted  ee  to  eB  ,  arriving  at  : 


This  equation  for  a  fixed  parameter  6b  ( =  10  3)  and  for 
each  value  of  q  gives  a  value  of  u.  To  plot  a  utility  curve 
one  also  needs  the  value  of  D  as  a  function  of  the  number  of 
frequencies  (q).  Arthurs  and  Dym  recommend  a  bandwidth 
law  for  this  purpose  (their  equation  133)  which,  in  the  magni¬ 
tudes  of  this  paper,  can  be  expressed  as  : 
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0  =  2.-^  =  ^. _L_  .  (40) 

2q  +  1  q  1+  l/2q 

With  Hie  help  of  equations  39  and  40  it  is  now  possible 
to  plot  curves  0A  and  GB  as  upper  and  lower  bounds  of  the 
utility  of  an  ideal  coherent  MFSK  system.  The  special 
case  q  =  S'  leads  to  u  =  —9.9  db  and  1)  =  —4.0  db,  a  point 
which  lies  also  on  curve  12A,  the  utility  curve  for  the  ideal 
binary  coherent  FSK  system.  This  point  is  the  starting  point 
for  both  the  bounds  of  equation  39.  The  utility  curves  GA 
and  GB  reach  the  highest  bit  density  for  a  quaternary  MFSK 
system  and  the  utility  monotonously  increases  with  the 
increasing  size  of  the  MFSK  waveform  alphabet.  The  hit 
density,  however,  rapidly  declines  beyond  octonary  systems. 
Comparing  nonbinary  FSK  with  binary  FSK  shows  that  the 
ideal  nonbinary  system  offers  at  least  7-db  improvement  in 
utility  over  the  binary  system  when  using  128  frequencies 
and  reading  the  value  from  the  lower  bound  of  the  utility 
curve.  Tile  upper  bound  of  the  utility  curve  is  represented 
by  curve  GA  and  results  from  the  lower  bound  of  equation  39. 
This  utility  curve  apparently  is  overly  optimistic,  since 
we  see  that  it  crosses  the  upper  bound  of  the  channel  capacity 
at  D  =  -15  db.  The  lower  bound,  curve  GB,  was  derived 

earlier  by  Helstrom  (GO)  and  it  falls  well  within  the  range 
of  Akima’s  model. 

Curves  5A  and  5B  relate  to  another  model  of  a  coherent 
MFSK  system  which  differs  only  slightly  from  the  Arthurs 
and  Dym  model.  It  is  based  on  the  “  theory  of  the  m- 
positions  experiment  ”,  resulting  in  an  integral  for  the  proba¬ 
bility  of  success  of  the  experiment  (Lawson  and  Uhlenbeck  50). 

Using  tables  or  computers,  one  arrives  at  error  curves 
of  the  form  shown  by  fig.  1  in  Akima’s  paper  (Akima  03). 
Akima’s  charts,  like  most  of  the  calculations  and  charts  in 
the  literature,  are  for  the  q-aiy  digital  signal  error  probability 
and  for  the  digital  energy.  We  converted  Akima’s  charts 
to  bit-error  probability  (c«),  using  for  the  first  purpose  both 
limits  of  equation  2.  The  result  is  plotted  in  (ig.  1 2  as  area  5A. 
The  curves  {indicate  that  the  operat  ion  of  the  decoding  system 
has  no  great  influence  on  the  utility.  Whether  the  decoder 
makes,  on  the  average  for  each  m-ary  error,  only  one  binary 
error  or  m/2  binary  errors  (equation  2)  changes  the  utility 
curve  only  from  the  upper  limit  of  area  5A  to  the  lower 
limit  of  area  5 A. 

When  plott  ing  the  curves  5A,  we  used  Akima’s  assumption 
about  the  band  occupancy  in  MFSK  systems.  His  assump¬ 
tion  leads  to  the  following  equation  for  the  bit  density  : 


q 

It  can  bo  seen  that  his  is  a  special  case  of  equation  7G  of 
Arthurs  and  Dym  for  the  value  q  —  oo.  The  two  equat  ions 
differ  most  for  q  =  2  ;  this  is  the  reason  that  Akima  curves 
start  about  I  db  to  the  right  of  the  Arthurs  and  Dym  curves. 
For  large  qj- values  tho  Akima  curves  fall  excellently  above 
and  below  the  lower  bound  of  the  Arthurs  and  Dym  curves. 
Lieberman  (61)  used  the  same  mathematical  model  for  the 
investigation  of  a  closely  related  problem  and  Helstrom 
(GO)  give  an  I  approximate  formula  for  the  error  probability  of 
coherent  MFSK  systems  which  is  identical  with  Arthurs 
and  Dvm’s  lower  bound  (GB). 

Akima  also  has  a  family  of  error  curves  for  an  MFSK 
system  with  noncoherent  detector.  His  equation  is  based 
on  original  work  by  Reiger  (58)  and  Lieberman  (G 1 ).  Akima’s 
curves  for  noncoherent  MFSK  appear  as  the  utility  area 
5B  in  our  fig)  12.  Again  we  used  both  limits  of  our  equation  2 
when  converting  tho  results  from  digit  error  ratio  to  bit 
error  ratio.  It  is  significant  that  the  difference  of  the  utility 
of  a  coherent  MFSK  system  as  compared  with  a  noncoherent 
MFSK  system  is  only  about  1  db.  This  value  is  practically 
independent  of  q,  the  number  of  frequencies. 

Looking  at  all  the  ideal  models  of  MFSK  systems  in  fig.  12, 
one  wonders  if,  at  least  theoretically,  such  systems  could 
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reach  Shannon’s  upper  bound  (curve  1)  when  q  is  going  to 
infinity.  This  is  indeed  the  case.  Turin  (59)  has  shown 
that  the  utility  approaches  the  value  —log,  2  for  D  — >  0  in 
all  orthogonal  systems  with  optimum  coherent  detection,  no 
matter  what  error  ratio  is  desired.  Thus  it  has  been  proved 
that  the  curves  5A  and  GB  will  asymptotically  approach 
curve  1.  But  to  achieve  this  goal  one  has  to  use  unrealisti¬ 
cally  high  q  values.  Jacobs  (G3)  showed  that  the  same 
asymptotical  behaviour  holds  for  noncoherent  MFSK 
systems  ;  Splitt  (G3)  independently  confirmed  these  results 
and  showed  that  the  same  asymptotic  approach  also  holds 
for  multiple  time  shift  keyed  (MTSK)  systems. 

All  these  mathematical  models  assumed  a  coherent  trans¬ 
mitter,  a  matched  filter  extractor  (either  lor  the  complete 
waveform  or  the  envelope  waveform)  and  a  maximum  likeli¬ 
hood  decider.  No  special  waveform  must  be  assumed,  but  a 
rectangular  envelope  and  a  constant  frequency  and  phase 
throughout  a  digital  interval  is  the  logical  choice.  Waveform 
distort  ions  and  cross  talk  between  the  q  frequencies  are  not 
considered.  Ideal  synchronization  is  assumed. 

These  idealizations  never  exist  in  practice.  For  binary 
coherent  and  noncoherent  FSK  systems,  Stein  (G4)  derived 
expressions  for  the  error  ratio  under  full  consideration  of 
the  cross  talk  due  to  non-ideal  filters  and  under  the  assump¬ 
tion  of  partially  correlated  noise  components  at  the  output 
of  the  filters.  Stein’s  paper  may  serve  as  a  starting  point 
for  the  derivat  ion  of  similarly  improved  mat  hematical  models 
of  MFSK  systems.  Any  improvements  of  this  type  have  to 
start  with  the  detailed  evaluation  of  the  spectra  of  MFSK 
waveforms.  The  rectangular  envelope  of  the  digital  signals 
is  not  necessarily  the  optimum  waveshape  ;  it  was  merely  a 
convenient  starting  point  for  the  first  models.  Sunde  (59) 
published  a  very  detailed  investigation  of  this  problem  of 
ideal  pulse  transmission,  comparing  the  characteristics  of 
AM  and  FM  for  t  his  purpose.  Although  his  paper  is  restricted 
to  tho  binary  case,  his  conclusions  still  apply  to  the  individual 
waveforms  of  a  MFSK  system.  These  conclusions  indicate 
that  intersymbol  interference  can  be  avoided  in  FM-type 
transmissions  without  the  need  of  a  wider  filter  band  than 
in  double-sideband  AM,  when  applying  partial  pulse  shaping 
by  premodulation  and  post-detection  low-pass  filters.  This 
fundamental  work  of  Sunde  was  further  amplified  by  the 
previously  ment  ioned  research  of  Bennett  and  Rice  (63a)  that 
included  the  case  of  a  discont  inuous  phase  at  the  transitions 
between  the  digital  elements.  The  papers  by  Salz  and  Stein 
(64b)  and  by  Tjhung  (64)  discuss  the  spectra  of  telegraphic 
characters  in  FSK,  the  distribution  of  the  instantaneous 
frequency  for  signal  plus  noise  in  q-ary  FM,  and  the  band 
occupancy  of  digital  FM.  A  short  report,  of  Kobos  (65) 
may  offer  some  hints  about  codes  for  *•  quantized  frequency 
transmission  ”  (QFT),  which  is  applied  in  military  communi¬ 
cations  systems.  As  far  as  the  transmission  system  is  con¬ 
cerned,  QFT  is  simply  another  name  for  MFSK. 

Any  attempt  to  improve  the  mathematical  models  of  the 
receiving  system  must  begin  with  the  non-ideal  character 
of  the  detection  system  itself.  The  models  leading  to  the 
utility  curves  of  fig.  12,  Nos.  5  and  G,  assume  either  ideal 
envelope  detector  or  ideal  coherent  demodulator.  The 
deficiencies  of  the  first  variety  have  been  demonstrated 
excellently  by  Fubini  and  Johnson  (48).  The  deficiencies 
of  the  second  variety  have  only  more  recently  come  under 
detailed  investigation.  We  showed  in  fig.  11  that  for  the 
binary  receiver  three  different  kinds  of  synchronous  (or 
coherent)  demodulators  are  applicable.  The  same  applies 
to  MFSK  systems. 

In  the  first  case  one  applies  for  each  signal  frequency  a 
separate  local  oscillator  at  the  constant  centre  frequency  of 
the  respective  bandpass  filter  (R2  in  fig.  10).  These  local 
oscillators  must  all  be  synchronized  from  a  master  timer. 

For  coherent  reception  a  phase-lock  loop  is  applied  (Rs 
in  fig.  10).  Many  investigations  have  been  made  in  the  last 
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years  of  the  performance  of  phase-lock  loops,  primarily  for 
the  demodulation  of  analog  signals  or  for  the  acquisition  of 
carrier  frequencies  in  space  communications  systems  (Fili- 
powsky  and  Muehldorf  05a)  or  in  tracking  and  navigation 
systems  (Jaffeo  and  Reehtin  55;  Viterbi  03a,  b;  Taus- 
worthe  00). 

(13.  MFSK  Systems  with  Bandpass  Extractors  ( Small 
Signal  liose) 

These  systems  are  modelled  after  the  binary  receiver 
arrangement  R,  or  R.  in  lig.  10.  whereby  the  noncoherent 
receiver  R,  is  preferred.  The  receiver  R2  would  need  a 
separate  oscillator  for  each  frequency  or  a  complex  frequency 
synthesis  system  to  replace  individual  oscillators.  The 
mathematical  models  5A  and  5B  show  that  the  theoretical 
advantage  to  be  bought  with  this  considerable  increase  in 
circuit  complexity  is  an  improvement  in  utility  of  only  1  db. 
Channel  instabilities  (fading,  phase  distortions,  frequency 
displacements)  are  likely  to  reduce  this  small  advantage 
even  further. 

Jordan  et  al  (55)  operated  an  experimental  MFSK  system 
of  this  kind.  They  operated  it  in  connection  with  a  teletype¬ 
writer  and  obtained  best  results  with  a  seven-frequency 
MFSK  system.  They  used  a  pair  of  pulses  to  express  each 
of  the  32  teletypewriter  characters,  allowing  each  pulse  to 
use  one  of  the  following  seven  tone  frequencies  :  1778, 

184!),  1923,  2000,  2080,  2103,  2250  Hz.  The  system  operated 
over  voice  channels  occupying  a  bandwidth  from  1740  to 
2290  Hz,  or  a  total  of  550  Hz.  It  operated  at  the  rate  of  a 
standard  teletypewriter,  which  needs  153  ms  for  one  character 
of  5  hits  (a  rate  of  30.0  bps).  The  result  is  a  bit  density  of 
I)  0.057  or  12.5  db.  This  operating  point  is  entered 
as  413  in  lig.  12.  using  the  author’s  power  contrast  figures. 
As  a  controlled  experiment  the  authors  operated  the  same 
equipment  in  the  binary  mode  and  obtained  results  that  are 
plotted  as  line  4A  in  fig.  12,  with  the  upper  end  of  the  line 
representing  the  reported  peak  performance.  All  values 
are  for  a  binary  error  ratio  of  10  :l  and  for  laboratory  tests 
with  white  noise  in  the  transmission  band.  During  the  tests, 
filter-shaping  effects  and  the  influence  of  simulated  pulsive 
disturbances  were  investigated. 

The  question  arises  if  there  is  an  optimum  number  (q„,,() 
of  frequency  levels  for  achieving  the  highest  hit  density. 
Khanovich  (04)  investigated  this  problem  and,  for  an  opera¬ 
tion  with  low  energy  contrast  (relatively  high  error  ratio), 
reaches  the  same  conclusion  that  we  indicated  for  the  ideal 
systems  :  For  a  constant  bandwidth,  the  transmission  rate 
has  a  rather  flat  maximum  around  q  4.  Khanovich  also 
shows  that,  for  systems  with  higher  energy  contrast  (smaller 
utility  and  smaller  error  ratio),  this  maximum  shifts  to 
higher  q  values,  but  always  remains  quite  flat. 

In  section  GO  we  shall  see  that  the  MFSK  systems  are 
preferred  to  MI’S K  or  MASK  systems  for  operating  over 
fading  channels.  In  this  connection  it  is  important  to  have 
decision  systems  which  can  operate  independently  of  the 
absolute  output  level.  Thomas  (60)  reports  an  improved 
decision  technique  that  is  applicable  to  systems  with  band¬ 
pass  extractors.  Although  Thomas  describes  his  improve¬ 
ment  in  connection  with  a  binary  FSK  system,  il  may  bo  of 
interest  to  MFSK  designers.  Wards’  (62)  short  note  on 
the  probability  of  error  for  “  largest-of  selection  ”  could  be 
helpful  when  transferring  such  practical  or  analytical  results 
from  binary  to  nonbinary  receiving  systems, 

(14.  MFSK  Systems  with  Matched  Filter  Extractors  (Large 
Signal  Base) 

The  theoretical  superiority  of  MFSK  systems  with  matched 
filter  extractors  has  been  analytically  demonstrated  by 
Reiger  (58).  Recently  Glenn  (66)  investigated  wide-band 
MFSK  systems  where  the  signal  base  for  each  of  the  wave¬ 
forms  is  larger  than  10,  while,  in  the  models  represented  by 
curves  5  and  6  in  fig.  12,  a  signal  base  of  one  had  been  assumed 


for  the  individual  waveforms.  We  plotted  in  fig.  1 1  the 
values  of  Glenn’s  charts  for  a  bit  error  ratio  of  10  3  and  a 
f3s  --  100.  As  there  are  many  more  design  difficulties  for 
matched  filters  in  a  carrier  band  than  for  low-pass  matched 
filters,  Glenn  uses  a  receiver  arrangement  where  each  MFSK 
subchannel  has  first  a  bandpass  filter  of  the  bandwidth  of 
the  waveform  assigned  to  this  subchannel.  This  is  followed 
by  an  envelope  detector  and  by  the  actual  matched  filter  as  a 
low-pass  filter.  The  outputs  of  these  filters  in  samples  are 
compared  in  a  maximum  likelihood  decider.  If  a  system 
of  this  kind  is  to  operate  with  high  utility,  the  envelope 
detectors  must  operate  at  or  below  0-db  power  contrast, 
Glenn  discusses  this  problem  of  small  signal  detection  in  full 
detail. 

The  advantage  of  such  wide-band  MFSK  systems  is 
primarily  in  the  area  of  anti-jam  systems  and  security 
systems,  as  it  is  difficult  to  detect  at  all  the  presence  of  wide¬ 
band  signals  that,  at  the  receiver  location,  have  a  much 
smaller  spectral  density  t  han  the  white  noise.  The  detection 
problem  of  such  systems  is  related  to  the  problem  of  radar 
signal  detection,  which  elicited  much  interest  in  the  1940s 
and  is  treated  in  a  special  monograph  (Marcum  and  Swel  ling 
60). 

A  system  with  the  matched  filters  directly  in  a  carrier 
band  of  720  kHz  bandwidth  is  the  “  Orthomatch  ”  system 
of  the  Lincoln  Laboratory  of  the  MIT,  Lexington,  Massa¬ 
chusetts  (Kuhn  et  al  63,  Kuhn  64).  Operating  at  a  rate  of 
160  Kbps,  its  calculated  and  measured  utility  curves  are 
shown  as  curves  No.  9  in  fig.  12  by  a  solid  line  and  by  a  line 
of  crosses,  respectively,  it  is  amazing  how  close  these 
values  approach  the  curves  of  the  ideal  system.  The 
same  laboratories  designed  another  MFSK  system  as  a 
military  satellite  communications  system  (Maguire  65).  The 
anti-jamming  feature  of  band-spread  systems  is  again  the 
principal  reason  that  the  system  operates  over  a  bandwidth 
of  20  MHz  somewhere  in  the  X-band  (around  8  GHz). 
Known  as  the  Lincoln  Experimental  Terminal  (LET),  it 
operates  with  the  Lincoln  Experimental  Satellite  (LEX). 
LES-2  was  launched  May  6,  1965.  into  a  synchronous  orbit. 
The  primary  purpose  of  the  system  is,  according  to  I.  L. 
Lebow  of  the  Lincoln  Laboratory,  “the  communication  of 
vocoded  speech  in  a  hostile  environment  ”  (Lebow  05).  LET 
is  designed  to  work  with  both  active  and  passive  satellite 
repeaters.  Vocoded  speech  is  speech  converted  to  binary 
digits  with  the  help  of  a  device  called  VOCODER  (voice 
coder  and  decoder).  The  out  put  of  the  digital  VOCODER  is 
available  in  9606  bps  or  4800  bps  data  streams.  The  MFSK 
transmission  system  operates  with  q  16,  j3,s  I  and  a 
digital  rate  of  5000  digits  per  second  (dps).  Thus  t  he  system 
can  handle  5000  .  log,  16  20,000  bps,  i.e.,  more  than  two 

times  or  four  times  the  VOCODER  output  rate.  This 
redundancy  is  used  for  a  sequential  coding  and  decoding 
system,  which  ensures  practically  error-free  operation 
(Lebow  65).  The  jam-resistance  of  the  system  results  from 
the  fact  that  the  20-MHz  transmission  band  is  subdivided 
into  4096  subbands  of  about  5-KHz  bandwidths  each. 
Only  16  of  these  bands  are  selected  at  any  one  time  for  use 
as  the  16  MFSK  subchannels.  The  selection  rules  are  known 
to  all  participating  stations  and  are  changed  frequently. 

From  the  available  data  and  with  the  help  of  a  few  assump¬ 
tions  by  this  author,  the  utility  lines  for  this  system  may 
be  placed  approximately  at  the  positions  of  lines  10A  and  10B 
in  tig.  12,  assigning  as  total  bandwidth  merely  the  bandwidth 
of  16  subchannels  and  not  of  4096  subchannels.  Line  10A 
corresponds  to  the  4800-bps  rate  and  line  10B  to  9600-bps 
rate. 

The  characteristics  that  make  matched  filter  MFSK  systems 
jam-resistive  are  also  desirable  characteristics  when  operating 
over  heavily  dispersive  media,  such  as  the  use  of  an  orbiting 
dipole  belt  (project  Westford)  for  communications  (Lebow 
et  al  64).  This  kind  of  application  will  be  discussed  further 
in  subsection  G6. 
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G5.  Digital  FM  Systems 

The  examples  of  binary  FSK  systems  in  fig.  10  showed 
combinations  of  T3  with  R„  R^  or  R0 ,  which  are  essentially 
analog  FM  systems  transmitting  a  binary  waveform  and 
adding  a  sheer  at  the  output.  The  sheer  can  be  replaced 
by  a  clocked  resampler  and  a  decider  to  restore  the  accurate 
binary  elementary  length  of  each  element.  When  feeding  this 
system  with  a  multilevel  low-pass  waveform  (since  it  is  the 
result  of  a  video  MASK  system),  the  total  system  becomes  a 
special  variety  of  an  MFSK  system  which  best  may  be 
designated  as  MASK-FM  (soction  F2). 

In  fig.  11  we  showed  that  the  systems  of  this  class  operate 
at  the  right  side  of  the  utility  chart,  similar  to  the  MASK 
systems.  Their  advantage  over  MASK  systems  is  their 
insensitivity  to  variations  in  attenuation  of  the  channel 
(fading)  or  gain  of  the  amplifiers.  To  compete  in  bit  density 
with  MASK  systems,  digital  FM  systems  are  required  to 
operate  as  small-bund  FM  systems. 

An  experimental  digital  FM  modem  (modulator-demodula¬ 
tor  set,  or  data  terminal)  was  developed  in  the  Bell  Tele¬ 
phone  Laboratories  and  described  by  Salz  and  Koll  (66). 
Wo  plotted  the  results  of  Salz  and  Kofi's  laboratory  tests 
with  line  simulators  and  noise  generators  in  fig.  12,  area  8. 
The  opera, ing  point  8A  is  for  binary  operation;  8B,  for 
duobinary  operation  (duobinary  MASK  signals  transmitted 
by  FM)  ;  and  8C,  for  quaternary  operation  (four-level  MASK 
transmitted  by  FM).  The  lino  from  8A  to  8C  represents 
the  change  from  q  =  2  to  q  =  4. 

A  number  of  related  investigations  should  be  mentioned 
to  assist  development  engineers  who  intend  to  explore  further 
this  interesting  field  of  digital  FM  (or  quantized  FM,  as  it 
is  frequently  called).  Lender’s  (65b)  paper  on  binary 
orthogonal  FM  techniques  was  mentioned  in  section  Gl.  It 
wifi  be  helpful  to  clarify  transmitter  and  receiver  problems 
for  MASK-FM.  Bennett  and  Salz  (63b)  disclosed  the  original 
analysis  underlying  the  development  of  the  BTL  digital 
FM  terminal  discussed  previously.  Of  part  icular  importance 
is  the  theory  of  digital  FM  spectra.  Distortions,  transmission 
bandwidth,  and  peak  deviation  are  all  key  parameters  in 
determining  the  bit  density  of  digital  FM  systems.  Sunde’s 
1959  paper  became  a  standard  work,  which  later  was  ex¬ 
panded  by  Bennett  and  Rice  (63a)  to  an  analysis  of  tho  spect¬ 
ral  density  and  autocorrelation  functions  associated  with 
binary  FSK.  Building  on  this  analytical  background, 
Anderson  and  Salz  (65b)  investigated  the  spectra  of  digital 
FM  of  four  and  eight-level  signals,  and  Salz  (65a)  derived 
compact  formulas  for  the  spectral  density  function  of  an 
ensemble  of  continuous-phase  constant-envelope  multilevel 
FM  waves.  Independently  of  the  above  authors,  Pelchat  (64) 
arrived  at  an  expression  for  the  power  spectrum  and  the 
autocorrela  ion  function  of  PCM-FM  waves. 

An  interesting  possibility  for  digital  FM  detection  has 
been  analyzed  by  Anderson  et  al  (65)  for  the  binary  case. 
Called  the  differential  detection  method,  it  involves  the 
detection  of  binary  FM  by  multiplication  of  the  received 
signal  by  itself,  delayed  for  one  elementary  length. 

From  Germany  comes  a  report  of  laboratory  tests  of  a  four- 
level  digits  FM  set  (Von  Hiinisch  and  Kettel  63).  The 
results  yield  the  operating  point  11  in  fig.  12  for  tho  four-level 
set.  This  is  close  to  the  binary  FM  operating  [joint  of  the 
BTL  experimental  digital  FM  terminal,  while  the  correspond¬ 
ing  binary  operating  point  of  the  German  set  is  about  2  db 
lower  anti  2  db  to  the  loft. 

G6.  Special  Applications  of  MFSK 

FSK  systems  are  generally  praised  for  their  relatively 
good  perfor:  nance  under  atmospheric  or  pulsive  noise  distur¬ 
bances.  Bello  (65b)  compared  an  FSK  modem  with  16 
subchannels  operating  in  parallel  with  a  quaternary  PSK 
modem,  showing  a  slight  advantage  (about  3  db)  in  utility 
for  the  FSK  system  under  HF  radio  transmission  conditions. 
Mrs.  Conda  (65)  analyzed  tho  behaviour  of  a  noncoherent 

408  Proceedings  I.R.E.E.  Australia 


FSK  system  (NCFSK)  under  atmospheric  noise  conditions. 
She  provides  interesting  design  charts  that  should  bo  of  great 
assistance  to  systems  planners  of  such  links.  The  effect  of 
impulse  noise,  like  that  encountered  in  telephone  voice 
circuits,  on  FSK  systems  is  tho  subject  of  two  investigations. 
Bodonyi  (61)  found  that  noncoherent  FSK  systems  yield 
the  lowest  error  ratio  of  all  binary  systems  compared  under 
the  influence  of  heavy  impulsive  disturbances.  Engel  (65), 
however,  gave  a  different  rating,  finding  in  the  presence 
of  impulsive  noise  no  different  ranking  of  the  competitive 
digital  transmission  methods  than  under  Gaussian  noise 
conditions.  His  results  leave  the  FSK  system  os  the  second 
worst,  being  inferior  to  SSB,  DSB,  and  PSK,  and  being 
superior  only  to  AM  with  envelope  detection.  He  used  a 
binary  FM  system  with  frequency  discriminator. 

Although  the  previously  mentioned  tests  applied  to  binary 
FSK  systems,  it  is  evident  that  their  results  are  highly 
indicative  of  the  behaviour  of  MFSK  systems  under  pulsive 
noise  conditions.  More  direct  investigations  of  the  perfor¬ 
mance  of  MFSK  are  available  in  connection  with  the  trans¬ 
mission  over  fading  channels  and  particularly  in  connection 
with  diversity  systems. 

Jordan  (61)  compared  the  various  anti-multipath  methods 
known  at  that  time.  He  concludes  that  multilevel  FSK 
wifi  require,  under  certain  circumstances,  less  penalty  in 
transmission  rate  for  anti-multipath  measures  than  binary 
FSK.  Servinskiy  (64),  Dumanian  (63),  and  J.  N.  Pierce  (66) 
offer  curves  that  may  help  in  the  design  of  “  parallel  FSK 
channels  ”  (MFSK)  to  find  the  best  compromise  between 
complexity,  power,  and  bandwidth  when  the  digital  length  To 
is  prescribed  because  of  the  encountered  multipath  conditions. 

The  advantages  of  diversity  reception,  have  been  known 
for  some  time  (Van  Wambeck  and  Ross  51 ).  J.  N.  Pierce  (58) 
analyzed  methods  of  using  diversity  reception  to  improve 
FSK  systems  when  operating  over  Rayleigh  fading  channels. 
The  same  author  (J.  N.  Pierco  61)  amplified  his  investigations 
by  a  note  comparing  space  diversity  with  frequency  and 
time  diversity  when  transmitting  binary  signaLs.  The 
theoretical  diversity  improvement  in  MFSK  systems  is  the 
subject  of  an  analysis  by  Hahn  (62).  He  concludes  that 
MFSK  systems  can  be  improved  by  diversity  methods 
when  operating  in  the  presence  of  random  noise  and  fading. 
He  recommends,  as  the  optimum  combining  method,  that 
one  square  and  add  tho  detected  outputs  of  corresponding 
filters  from  each  diversity  channel  and  then  apply  the  maxi¬ 
mum  likelihood  decider. 

Kennedy  and  Lebow  (64)  gave  an  introduction  to  the 
principles  of  communications  over  dispersive  channels. 
They  developed  a  channel  model,  presented  simple  formulas 
and  curves  for  the  selection  of  communications  parameters 
for  operation  over  multipath  channels,  and  discussed  binary 
FSK  and  MFSK.  An  application  of  these  fundamentals 
to  tho  orbitting  dipole  bolt  channel,  a  highly  dispersive 
channel  resulting  from  project  “  Westford  ”,  is  discussed 
by  Lebow  et  al  (64)  and  Brookner  (65b). 

The  application  of  MFSK  systems  as  anti-jamming  systems 
or  secure  communications  systems  was  discussed  in  subsec- 
section  G4.  Here  we  may  mention  Culbertson  and  Perkins 
(64),  who  described  a  solid  state  digital-to-digital  converter 
for  use  with  high  frequency  SSB  transceivers,  utilizing  the 
principle  of  quantized  frequency  modulation  (QFM).  An 
airborne  flight  test  model  was  used  in  1963  in  test  flights, 
comparing  this  system  with  an  audio  frequency  shift  keying 
(AFSK)  system.  Another  interesting  application  of  MFSK 
is  a  random-access  communications  system  using  frequency 
shifted  pseudo-noise  (PN)  signals,  described  by  Blasbalg 
et  al  (64). 

Somewhat  related  to  MFSK,  but  better  classified  as  poly¬ 
signal  systems,  is  a  digital  data  transmission  model  originally 
described  by  Renshaw  (64).  Continuing  these  studies, 
Renshaw  (65b)  described  a  signal  processing  system  and  a 
delay  line  time  compressor  (DELTIC),  which  can  be  used 
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in  connection  with  the  nmltitone  parallel  FSK  system 
mentioned  above.  This  combination,  according  to  the 
author,  allows  much  larger  signalling  alphabets  without 
paying  the  penalty  of  excessive  equipment  complexity. 

Another  piece  of  related  equipment  is  an  experimental 
terminal  of  the  Bell  Telephone  Laboratories  (R.  C.  Anderson 
et  al  65).  Although  it  falls  into  the  class  of  polysignal  systems, 
it  is  mentioned  here  because  it  was  used  in  tests  over  a  cross 
section  of  the  various  types  of  facility  found  in  the  switched 
network.  The  set  transmits  data  in  binary  FSK  simul¬ 
taneously  over  nine  subbands  within  the  voice  band.  Re¬ 
lationships  between  error  performance  and  transmission 
characteristics  were  investigated. 

As  a  last  application  of  MFSK  techniques,  the  multi¬ 
frequency  signalling  system  of  the  dial  telephone  service 
should  be  mentioned.  The  receiving  part  has  much  in 
common  with  MFSK  receivers  (Pommerening  65).  The 
transmission  experience  over  the  switched  network  and  the 
error  rates  of  such  dial  signals  should  have  significance  for 
the  design  of  MFSK  modems  for  voice  circuits. 

In  conclusion,  it  should  be  noted  that  developments  arc 
progressing  on  the  largo  scale  that  should  be  expected  when 
looking  at  fig.  11,  which  shows  the  range  of  MFSK  systems 
extending  over  all  decades  of  bit  density. 

H.  Multiphase  Shift  Keying  (MPSK)  Systems 

HI.  Fundamentals  of  MPSK  Systems 

The  oldest  application  of  phase  modulation  for  digital 
t  ransmission  is  most  likely  a  kind  of  tone  telegraphy  which 
operates  in  a  bipolar  two-state  mode  as  a  double  sideband 
suppressed  carrier  (DSB-SC!)  system.  In  this  mode  a  Mark 
corresponds  to  one  phase  (or  positive  amplitude),  and  a 
Space  corresponds  to  the  opposite  phase  (180-degree  shifted 
or  negative  amplitude)  according  to  the  identity  : 

—  sin  wt  t  =  sin  (w,  t  +  w)  .  (42) 


This  DSB-SC  or  biphase  system  is  also  the  prototype 
for  the  ideal  binary  system  shown  as  curve  3  in  all  utility 
charts  (figs.  8  or  12).  Montgomery  (54)  established  the 
theoretical  superiority  of  such  a  biphase  system  over  FSK 
and  ASK  for  both  cases — no  fading  or  Rayleigh  fuding  of 
the  carrier. 

Doelz  (53)  suggested  a  four-phase  system  which  ho  pub¬ 
lished  later  in  more  detail  (Doelz  ot  al  57).  Since  then  it 
has  become  known  as  the  Kineplex  system  (subsection  H3). 
Loseo  (58)  compared  the  binary  and  quaternary  digital 
phase  modulation  techniques.  His  paper  reflects  the  opinion 
of  data  transmission  equipment  designers  on  FSK  at  that 
time.  Since  then  PSK  has  gained  the  attention  of  many 
laboratories  and  its  performance  is  well  understood,  both 
analytically  and  practically. 

PSK  is  credited  with  combining  the  advantages  of  both 
ASK  and  FSK,  without  suffering  from  the  shortcomings 
of  these  systems.  Although  this  statement  is  a  generaliza¬ 
tion  as  the  result  of  oversimplified  theories,  it  cannot  bo 
denied  that  PSK  requires  less  bandwidth  than  FSK  and  offers 
advantages  over  ASK  when  used  over  slowly  time-varying 
channels.  The  greatest  problem  with  PSK  is  the  require¬ 
ment  for  a  reliable  phase  reference  in  the  receiving  sot. 
The  many  varieties  of  PSK  systems  differ  in  the  methods 
used  to  establish  this  phaso  reference.  There  are  two 
major  classes  of  PSK  and  MPSK  systems.  The  first, 
called  the  automatic  phase  reference  (APR)  class,  transmits 
the  phase  reference  in  the  same  signals  that  carry  the  informa¬ 
tion  (subsection  H3).  The  second,  called  the  separate 
phase  reference  (SPR)  class,  provides  the  reference  by  means 
that  differ  from  the  information-carrying  signals  (subsection 
H4).  However,  a  discussion  of  the  fundamentals  of  the 
binary  PSK  systems  will  help  to  understand  the  details  of 
the  MPSK  systems,  described  later. 

Figure  13  shows  12  frequently  used  modes  of  carrying 
binary  signals  in  a  low-pass  channel  (video  channel  or  base 
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band  channel).  Waveform  1  is  the  most  usual  two-state 
mode  with  Space  corresponding  to  one  voltage  level  (0) 
and  Mark  corresponding  to  another  voltage  level  ( + 1 ). 
If  several  Marks  follow  each  other,  the  voltage  does  not 
return  to  kero  (NRZ)  (Bennett  and  Davey  65).  This  wave¬ 
form  apparently  has  a  dc  component  that  is  the  average 
voltage  of  the  waveform.  When  discontinuing  transmission, 
the  waveform  will  stop  in  either  state.  This  Is  not  the  case 
with  the  roturn-to-zero  (RZ)  waveforms  2  and  3,  although 
theso  waveforms  have  a  dc  component.  A  bipolar  non¬ 
return  waveform  (No.  4)  could  avoid  a  dc  component  in  the 
special  case  where  there  is  an  equal  number  of  Mark  and 
Space  symbols. 

The  suppression  of  a  dc  component  is  possible  through  the 
introduction  of  a  rectangular  subcarrier  of  a  fundamental 
frequency  exactly  equal  to  the  bit  rate.  The  information 
symbols  modulate  this  subcarrier  in  a  biphase  mode.  Wave¬ 
form  5  is  the  digital  subcurrier.  The  resulting  waveform 
(No.  6)  can  never  stay  longer  than  one  bit  length  in  tin;  same 
state  ;  i.e.,  no  dc  component  can  materialize. 

The  digital  diphase  (7)  is  the  differential  equivalent  of 
the  digital  biphase.  It  always  takes  the  phase  during  the 
previous  symbol  and  reference  and,  if  the  input  information 
element  is  a  Mark,  changes  the  phase  ;  if  it  is  a  Space,  it 
retains  the  same  phase. 

Naturally,  a  waveform  with  rectangular  elements  has  an 
unnecessarily  wide  spectrum.  Therefore,  RCA  introduced  a 
filtered  version  of  the  diphase  waveform,  which  it  calls 
“  modified  diphase  ”  (Douglas  et  al  61).  This  waveform 
is  shown  as  No.  8  in  lig.  13,  where  it  has  zero  crossings  at 
the  centre  of  each  bit  interval.  This  characteristic  is  very 
helpful  for  learner  extraction.  Line  9  in  fig.  13  shows  that  a 
resampling]  operation  at.  the  first  quarter  of  each  interval 
(i.e.,  at  the  instants  marked  1,  5,  9,  .  .  .)  af,  the  top  of  fig.  13 
is  a  good  Way  to  recover  the  information  at  the  receiver. 
Line  10  shows  that  the  modified  diphase  waveform  carries 
even  more  (information  about  the  received  message.  When 
resampling  at  the  separation  lines  bet  ween  any  two  message 
elements  (ije.,  at  the  instants  0,  4,  8.  .  .  .),  the  receiver  may 
detect  the  absolute  polarity  of  the  elements  without  any 


reference  to  the  previous  element.  If  the  sample  is  zero 
at  this  instant,  a  Space  is  the  message  element  carried  by  the 
following  interval.  If  a  positive  or  negative  pulse  is  the 
result  of  the  resampling  process,  a  Mark  is  carried  by  the 
following  element.  Line  1 1  finally  shows  another  form 
of  redundancy  and  its  utilization.  It  has  been  called  “  double 
sampling  detection  ”  (Atzenbeclt  65).  Each  message  interval 
is  now  resampled  twice,  at  the  one-quarter  point  and  at  the 
three-quarters  point  (i.e.,  at  the  instants  1,  3,  5,  7,  .  .  .). 
If  a  pair  has  the  same  sequence  of  polarity  as  the  previous 
pair,  a  Space  has  been  received  ;  if  the  sequence  of  polarities 
of  the  pair  of  pulses  differs  from  that  of  the  previous  pair,  a 
Mark  has  been  received.  If  the  two  pulses  of  a  pair  have 
the  same  polarity,  a  mutilated  element  has  been  received, 
as  this  is  a  “  forbidden  ”  combination  for  a  pair  of  resamples. 
Error-correction  schemes  or  ARQ  (repetitious)  operations 
can  then  act  to  avoid  an  output  error. 

Line  12  shows  another  mode  of  encoding  that  is  used 
occasionally.  It  is  designated  as  differential  no-return-to- 
zero  (DNRZ)  and  it  follows  the  rule  that  a  “  zero  ”  level  is 
produced  when  the  element  has  the  same  symbol  as  the  pre¬ 
vious  element,  and  a  “  one  ”  level  is  produced  if  it  has  the 
opposite  symbol.  The  weakness  of  this  waveform  is  obvious 
when  compared  with  digital  diphase. 

After  this  review  of  binary  formats,  it  will  be  easy  to  survey 
the  many  transmitter  and  receiver  arrangements  that  have 
been  devised  for  binary  PSK.  They  are  shown  in  fig.  14 
in  a  presentation  similar  to  the  binary  FSK  arrangements 
in  fig.  10.  MPSIv  systems  evolved  from  these  fundamental 
binary  arrangements.  Basically,  the  various  modes  of 
carrier  extraction  are  the  same  for  PSK  and  for  MPSK. 

There  are  four  transmitter  arrangements  and  nine  receiver 
arrangements.  As  in  the  FSK  case,  there  are  some  combina¬ 
tions  more  efficient,  than  others.  They  are  indicated  by 
shaded  areas  in  the  central  matrix. 

'I’he  transmitter  arrangement  T,  is  the  one  that,  many 
of  the  early  PSK  systems  used.  Tt  has  as  an  optional  feature 
the  generation  of  a  pilot  frequency  f/>  (Walker  65). 

The  transmitter  arrangement  Ts  is  applied  in  those  cases 
where  the  carrier  reference  information  is  transmitted  via 


Figure  II. — Circuit  arrangements  of  binary  PSK  systems. 
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tho  quadrature  component  of  the  subcarrier.  An  important 
design  problem  is  the  determination  of  the  fraction  of  the 
total  transmitter  power  which  should  be  assigned  to  tbe 
synchronization  channel.  The  adjustment  of  this  power 
percentage  can  be  made  with  the  help  of  an  attenuator 
(ATT  in  tig.  14,  T,  and  T2)  (Stiffler  64b). 

Tho  arrangement  T3  is  used  to  generate  the  previously 
mentioned  digital  biphase  signals  ;  and  T  ,  ,  the  diphase 
signals. 

The  receiver  arrangements  in  fig.  14  show  nine  varieties, 
each  of  which  differs  in  the  methods  applied  to  recover  the 
carrier  phase  and  to  establish  bit  synchronization. 

The  arrangement  R,  ,  well  known  from  radar  and  ranging 
systems,  is  applied  when  an  external  local  reference  is 
available.  The  rather  complex  synchronization  circuits 
used  to  achieve  bit  synchronization  are  not  shown  in  fig.  14, 
Rj  ,  but  a  good  presentation  of  these  problems  may  bo  found 
in  Hopner  (59). 

The  roceiver  arrangement  R2  is  based  on  the  fact  that  it  is 
easier  to  recover  the  doubled  carrier  frequency  than  tho 
carrier  frequency  itself.  Because  of  the  biphase  operation, 
the  information  is  consistently  in  phase  with  a  frequency 
equal  to  twice  the  carrier  frequency,  although  it  is  partly 
in  phase  and  partly  out  of  phase  with  the  carrier  frequency 
itself.  The  difficulty  with  this  arrangement  is  that  special 
measures  are  needed  to  determine  the  absolute  polarity,  which, 
naturally,  is  lost  in  the  full-wave  rectification  process  (Fisch 
63,  Critchlow  et  al  64). 

The  arrangement  Ra  is  generally  known  as  the  squaring 
loop  (Lindsey  66,  Stiffler  64a).  It  combines  tho  advantages 
of  the  full-wave  rectifier  with  those  of  a  phase-lock  loop 
for  carrier  frequency  recovery  ;  however,  it  also  requires 
external  means  to  determine  the  absolute  polarity  of  the 
received  signals. 

R ,  is  a  slightly  modified  version  of  the  squaring  loop  which 
operates  the  voltage -controlled  oscillator  (VCO)  at  twice 
the  carrier  frequency  (Hannan  and  Olson  61),  whereas,  in 
the  arrangement  Ra ,  the  VCO  operates  at  the  carrier 
frequency. 

The  arrangement  R5  shows  a  phase-lock  loop  with  feedback 
(PLLFB),  where  an  attempt  is  made  to  assure  the  correct 
polarity  by  remodulating  the  VCO  output  with  the  recovered 
information,  to  perform  in  the  synchronous  demodulator  (SD) 
a  complete  correlation  process  of  the  noisy  waveforms  with  a 
clean,  locally  generated  waveform  (Choate  60). 

R„  shows  the  typical  receiver  arrangement  for  binary 
phase  differential  reception.  It  requires  a  delay  element 
to  store  the  phase  of  each  elemontary  interval  during  ono 
interval  length  for  comparing  it  with  the  next  element. 
Glenn  (60)  compared  this  arrangement  with  other  binary 
transmission  methods  and  pointed  out  the  disadvantage 
that  the  reference  signal  itself  depends  on  the  input  signal 
level.  Thus  it  can  be  demonstrated  that  this  receiver 
arrangement  breaks  down  faster,  at  low  signal  levels,  than 
other  similar  arrangements.  It  has  been  used  in  a  radio 
modem  for  shortwave  reception  of  biphase  signals.  Kolarcik 
and  Paramithas  (63)  compared  its  performance  with  that  of 
an  integrate-and-dump  arrangement  as  used  in  Kineplex 
(subsection  H3).  It  is  also  used  in  an  AFC  modem  as  a 
phase-correlation  system  (Losee  58). 

The  receiving  arrangement  R-  ,  known  as  the  Swift 
receiver  (Clark  64),  evolved  from  R„  by  introducing  limiters 
into  the  delayed  branch  and  the  direct  branch. 

The  arrangement  R  g  is  used  in  a  subchannel  modem  of  a 
successful  radio  data  transmission  system  (Kathryn)  (Bello 
65).  It  operates  with  a  matched  filter  extractor  and  with 
an  integrate-and-dump  circuit,  similar  to  the  quaternary 
Kineplex  system  (Doolz  et  al  57). 

R,  is  the  typical  receiving  arrangement  for  the  previously 
discussed  diphase  system  (transmitter  T4)  (Douglas  et  al  61, 
Atzenbeck  65,  Kolarcik  and  Paramithas  63). 

Concluding  this  review  of  biphase  circuitry,  it  should  be 


stressed  that  no  all-inclusive  comparison  of  these  varieties 
of  arrangements  seems  to  bo  available  in  the  literature. 
This  is  only  natural,  since  the  requirements  of  various 
applications  are  quite  different  and  since  many  of  the  circuit 
innovations  are  still  under  intensive  analytical  exploration. 

Fundamental  considerations  for  tho  comparison  of  phase 
shift  keyed  data  transmission  systems  may  be  found  in  tho 
papers  by  the  Armour  Research  Foundation  (61b),  Khvoros- 
tenko  (64)  and  Bussgang  and  Loiter  (66). 


H2.  Ideal  MPSK  Systems  and  Practical  Tests 


Two  major  classes  of  MPSK  systems  were  mentioned — 
the  automatic  phase  reference  system,  and  tho  separate 
phase  reference  (SPR)  system.  In  this  subsection  wc  present 
the  ideal  mathematical  models  for  both  classes,  beginning 
with  SPR. 

We  again  show  a  utility  chart  in  fig.  15,  which  contains, 
for  reference  purposes  :  curve  1,  representing  the  ideal 
Shannon  system  discussed  in  Part  I  ;  curve  2,  displaying 
the  utility  curve  of  the  ideal  MASK  system  discussed  in 
subsection  Fl  in  this  part;  and  curve  3,  the  ideal  binary 
system  with  A  =  —  1  and  with  matched  filter  extractor, 
which  demonstrates  the  upper  limit  of  utility  for  any  con¬ 
ceivable  uncoded  binary  system  with  bit-by-bit  decisions. 

Arthurs  and  Dym  (62)  developed  an  upper  and  lower 
bound  for  the  error  probability  of  an  ideal  MPSK  system 
with  external  reference  (coherent  MPSK),  assuming  again 
an  optimum  extractor  (matched  filter  plus  synchronous 
detector)  and  an  optimum  decider.  We  converted  these 
bounds  into  bounds  for  tho  utility,  arriving  in  our  notation 
at  tho  equation  : 
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Tho  error  function  Q  (x)  has  been  defined  and  plotted  in 
fig.  8b.  Equation  43  is  similar  in  its  form  to  equation  39 
for  MFSK.  The  argument  of  the  error  function,  however, 
is  different,  as  is  the  equation  for  the  bit  density,  which  now 
(for  MPSK)  is  simply  as  follows  : 


D  = 


logs  q 

Pn 


(44) 


In  both  equations  q  is  the  number  of  phase  states  used 
in  the  system.  Tho  states  are  assumed  to  be  equally  distri¬ 
buted  over  2n  =  360°.  Pd  is  tho  signal  base  for  one 
digit.  The  waveforms  of  all  digits  are  assumed  to  have 
rectangular  envelopes  and  constant  phase  over  the  whole 
digital  duration.  It  can  be  seen  that  pi>  does  not  enter 
equation  43.  This  assumes  the  use  of  the  minimum  signal 
base  that  is  compatible  with  distortion-free  operation. 
For  ideal  rectangular  waveshapo,  this  signal  base  would  be 
much  larger  than  one.  On  the  other  hand,  there  is  the 
assumption  of  a  matched  filter  extractor  that  would  restrict 
the  noise  bandwidth  of  a  rectangular  signal  to  about  B« 
-  1/To  .  Thus  it  seems  that  for  an  ideal  mathematical 
model,  Pd  =  1  is  a  reasonable  assumption.  Efforts  to  develop 
more  advanced  mathematical  models  considering  spectral 
limitation  in  detail  will  bo  reviewed  below.  We  plotted 
the  utility  area  resulting  from  the  Arthurs  and  Dym’s  model 
as  area  4  in  fig.  15  for  eo  =  10-*  and  Pd  =  1. 

It  is  not  surprising  that,  under  the  given  assumption, 
both  curves  start  for  q  =  2  at  tho  value  u  =  —6.8  db,  which 
is  identical  with  the  value  for  tho  ideal  binary  system. 
Indeed,  for  q  =  2,  equation  43  becomes  identical  with  equa¬ 
tion  34.  The  upper  limit  of  t  he  utility  reaches  a  flat  maximum 
close  to  q  =  3  and  then  decreases,  generally  running  parallel 
to  the  ideal  Shannon  model  (curve  1).  This  maximum  at 
q  =  3  has  been  pointed  out  by  Khanovich  and  Bondarev 
(65)  and  also  by  many  other  authors.  It  is  possibly  the 
paper  by  Calm  (59)  which  shows  this  maximum  for  the  first 
time  in  fig.  4  of  his  paper.  There  it  appears  as  a  minimum. 
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All  curves  are  for  eg  =  10 
except  where  i  fated  otherwise. 


Legend 


(0  Shannon's  upper  bound 
Ideal  MASK  system 
(5)  Ideal  binary  MF  system  (A  -  -I) 

(7)  Coherent  MPSK  system  (upper  ond 

low.  r  bound),  Arthurs  &  Dym,  1962 
©  Coherent  MPSK.  Cohn,  1959 

Diff.  coherent  MPSK,  Arthurs  &  D>m,  1962 
©  AN/FGC  54  0  q  4,  DC-MPSK  modem 
under  flat  fading  conditions 
®  Hughes  HC -270 DDT,  q  4,  DC-MPSK  modem 
over  co.  250  miles  leased  lines  in  Calif. 
©  The  same  as  but  over  Army  carrier  system 
(lO)  German  4-phase  modems  over  telephone  lines 
with  impulse  noise 

(jj)  AN-GSC  4  (Kineplex)  DC-MPSK  q  8  over 
leased  lines  (530  miles  of  H  44  coble) 

^2)  Diphase  modem  over  wideband  channel 
(130  kbps  over  400  kHc) 


Figure.  1 5. — Utility  curves  of  MPSK  systems. 


due  to  the  different  kind  of  normalization.  Calm’s  curve  of 
the  coherent  MPSK  system,  as  plotted  in  our  notation, 
falls  exactly  between  the  upper  and  lower  bound  of  the 
Arthurs  and  Dym  model  (curve  5,  fig.  15).  Notice  that 
Calm  does  not  use  a  matched  filter  extractor  but  resamples 
the  phase  at  the  peaks  of  the  received  waveform.  Yet, 
for  /}„  1,  both  methods  show  the  same  extraction  efficiency, 

because  the  efficiency  of  the  MF  extractor  is  proportional 
to  the  TB  product  (fin)  of  the  signals.  Again,  one  must 
remember  that  those  models  are  based  on  rather  ideal 
assumptions  and  that  neither  system  will  be  free  of  inter¬ 
symbol  interference  when  operating  with  Pi>  -  I .  Practical 
systems  will  show  a  difference  between  the  two  cases. 

Mathematical  models  are  also  available  for  the  other  class 
of  MPSK  systems,  the  APR  class,  i.e.,  the  systems  with  auto¬ 
matic  phase  reference.  Although  the  noisy  signal  is  com¬ 
pared  with  a  clean  reference  in  the  SPR  mode,  it  is  evident 
that  in  the  APR  mode  two  noisy  signals  are  compared  with 
each  other.  We  may  say,  in  a  crude  fashion,  that  there  is 
twice  as  much  noise  present  and  therefore  a  3-db  degradation 
is  to  bo  expected.  It  will  be  seen  that,  this  is,  indeed,  the 
cose  for  q  >  4. 

Most  APR  designs  merely  use  the  phase  difference  between 
a  pair  of  digits  transmitted  immediately  one  after  the  other, 
although  other  designs  involving  more  digits  are  conceivable. 
This  limit  to  the  phase  difference  between  two  digits  gave 
the  system;  of  this  class  the  name  differentially  coherent. 
Arthurs  and  Dym  (62)  gave  an  approximate  formula  for  the 
APR  case  (equation  82,  p.  353  in  their  paper).  In  our 
notation  this  can  be  written  as  follows  : 
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The  bit  density  for  this  model  is  given  by  equation  44. 
as  in  the  previous  case.  Equation  45  is  plotted  as  curve  6 
for  c/;  =  10" 3  and  for  q  values  from  4  to  64.  It  is  not 
accurate  enough  for  q  =  2.  For  this  extreme  case  (binary 
differentially  coherent  PSK),  Arthurs  and  Dym  gave  an 
explicit  formula  which,  in  our  notation,  can  be  written 
directly  in  decibels  ns  follows  : 


u1*6  =  — 10  log10  (-log,  2e,i)  .  (46) 

Comparing  equation  46  with  equation  3,3a  of  subsection  G2 
shows  that  the  ideal  binary  differentially  coherent  PSK 
(DC-PSK)  system  is  3-db  better  than  the  ideal  noncoherent 
FSK  system.  That  is  true  for  all  error  ratios  (ca)  ami  has 
been  stressed  by  many  authors  (Cahn  59,  Lawton  58). 
Salz  (65b)  compared  the  ideal  differentially  phase-coherent 
system  with  the  previously  discussed  digital  FM  systems 
and  reached  the  conclusion  that,  for  large  q  values,  both 
systems  have  nearly  the  same  utility.  For  q  =  64,  Salz 
found  that  digital  I’M  is  only  0.712  db  below  DC-PSK. 
The  error  statistics  of  DC-PSK  systems  are  important. 
It  is  reasonable  to  assume  that  errors  may  occur  in  pairs 
because  an  error  in  one  digit  will  also  base  the  next  decision 
on  an  unreliable  reference.  Salz  and  Saltzberg  (64)  analyzed 
this  phenomenon  and  showed,  for  additive  Gaussian  noise, 
that  the  ratio  of  double  errors  to  single  errors  is  loss  than 
0.25  when  the  SNR  is  small,  and  that  the  ratio  approaches 
zero  for  large  SNR.  Experimental  tests  verified  this  result 
but  also  showed  that  most  errors  occur  in  isolated  pairs 
when  isolated  noise  pulses  are  the  principal  disturbance. 
Bussgang  and  Loiter  (63)  provided  another  significant 
improvement  of  the  theory  of  DC-PSK  systems.  They 
used  decision  theory  to  improve  the  accuracy  of  Arthurs 
and  Dym’s  equation  81  in  the  range  where  it  is  most  inac¬ 
curate  (q  <!  4).  They  arrived  at  an  exact  expression  for 
q  =  4. 

Another  assumption  of  the  ideal  systems  which  is  subject 
to  improvement  is  the  width  of  the  decision  threshold. 
The  original  theories  reviewed  above  assumed  that  the  deci¬ 
sion  threshold  will  bo  placed  at  the  optimum  level  and 
that  the  maximum  likelihood  decision  (MLD)  rule  will 
be  applied.  This  includes  the  assumption  that  an  infinitely 
sharp  threshold  is  available.  Practical  circuits  cannot 
provide  such  an  ideal  situation.  Hubbard’s  (66)  paper  will 
therefore  be  welcomed  by  those  who  attempt  to  find  more 
realistic  approaches  to  mathematical  models.  He  analyzed 
the  effect  of  a  finite-width  decision  threshold  on  binary 
DC-PSK  systems.  This  analysis  may  be  extended  to 
MPSK  systems.  Another  significant  step  towards  realisti- 
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cal  mathematical  models  is  the  paper  by  Becker  et  al  ((54), 
which  analyzes  the  influence  of  frequency  offset  on  DC- 
MPSK  systems.  Lindsey  (Rob)  extended  the  analysis  of 
PSK  systems  to  include  the  effects  of  timing  errors  on 
the  detector  output  statistics.  He  indicated  the  deleterious 
effects  of  noisy  phase  references  by  graphical  representations. 
This  may  become  a  very  important  aspect  of  MPSK  theory. 

Interesting  contributions  to  the  theory  of  MPSK  systems 
came  from  Bussgang  and  Leiter  (64)  and  from  Proakis  et 
al  (64).  These  papers  deal  with  test  statistics  for  an  optimum 
DC-MPSK  system.  Leiter  and  Proakis  (64)  showed  in  a  short 
note  that  the  test  statistics  of  the  above  papers  were  actually 
equivalent. 

Despite  all  these  improvements  by  the  above-mentioned 
contributions,  there  is  one  major  deficiency  in  the  mathe¬ 
matical  models  :  the  consideration  of  the  bandwidth  occu¬ 
pancy.  In  only  a  few  applications  does  it  not  matter  how 
far  the  sidebands  of  a  MPSK  wave  range  into  other,  neigh¬ 
bouring  frequency  bands.  In  most  applications  there  is  a 
very  sharp  band  limitation,  which  has  to  be  enforced  by 
international  agencies,  military  frequency  control  personnel, 
or  simply  by  common  sense  of  all  the  users  of  a  wider  fre¬ 
quency  band.  In  any  case  it  is  clear  that  bandwidth  con¬ 
servation  (or  spectrum  conservation)  is  a  major  objective 
in  an  ago  of  tremendously  fast  expanding  communications 
facilities.  Yet  it  is  only  very  recently  that  the  engineering 
community  is  turning  its  interest  to  this  vital  problem. 
MPSK  systems  are  particularly  vulnerable  to  the  accusation 
that  they  unnecessarily  occupy  wide  frequency  bands, 
because  the  mathematical  models  of  the  systems  assume 
infinitely  steep  phase  transitions  at  the  end  of  each  waveform. 
MFSK  systems  are  better  in  this  respect  since  they  attempt 
to  operate  the  transmitters  with  coherent  waveforms  ; 
i.e.,  the  transition  from  one  frequency  to  another  is  estab¬ 
lished  with  continuous  phase.  There  is  no  discontinuity  in 
the  amplitude  at  the  transition  between  two  digital  elements. 
By  definition,  this  can  not  be  the  case  with  PSK  systems, 
though  the  enforcing  of  an  integer  ratio  between  subcarrier 
frequency  and  digital  transmission  rate  can  help  to  avoid 
excessive  discontinuities.  In  any  case  one  must  recognize 
that  t  he  spectral  distribution  of  the  energy  of  MPSK  waves 
is  a  major  problem,  deserving  careful  analytical  and  experi¬ 
mental  investigation.  In  this  sense  it  is  gratifying  to  review 
the  progress  over  the  last  few  years. 

Sunde  (61)  directed  the  attention  of  designers  to  the  prob¬ 
lem  of  the  spectral  distribution  when  comparing  different 
modulation  methods  and  their  susceptibility  to  phase 
distortions  in  the  transmission  channel.  He  pointed  out 
that,  although  PSK  has  an  advantage  of  a  few  decibels 
in  utility  over  FSK  under  ideal  conditions,  this  advantage 
will  fast  turn  into  inferior  performance  when  phase  distortions 
(delay  distortions)  affect  the  channel.  This  and  similar 
statements  of  other  authors  led  to  a  number  of  investigations 
of  the  spectral  distribution  of  phase  shift  keyed  signals. 
Rumble  (64)  compared  the  spectra  of  binary  PSK  systems 
under  the  various  coding  formats  presented  in  fig.  13.  Push- 
man  (63)  plotted  the  spectra  of  various  binary  modulation 
systems,  including  PSK  for  random  messages  ;  and  Karshin 
(64)  discussed  the  influence  of  the  IF  bandwidth  on  binary 
PSK  signals  for  various  transmission  rates.  Karshin  (66) 
points  to  the  strong  influence  of  the  TB  product  (|3«  value) 
of  the  individual  PSK  waveforms  on  the  error  probability 
of  binary  PSK  systems.  Petrovich  and  Razmakhnin  ((56) 
compared  spectra  of  MASK  and  MPSK  signals  and  concluded 
“  that  the  phaso  keying  spectrum  remains  practically  the 
same  as  the  number  of  phase  gradings  increases,  and  it  is, 
in  this  sense  analogous  to  the  spectrum  of  amplitude-keyed 
waveforms  ”.  Digital  computer  simulation  (Rappoport  65) 
turned  out  to  be  a  valuable  tool  when  investigating  the 
spectral  spread  of  quaternary  MPSK  signals,  and  also  their 
susceptibility  to  phaso  distortions  and  to  impulse  noise. 
Other  closely  related  contributions  come  from  Weber  (65) 


and  Pelehat  (65).  The  currently  available  practical  systems 
are  compared  in  the  utility  chart  of  fig.  15  with  the  older, 
rather  simplified  mathematical  models. 

Point  7  of  fig.  15  refers  to  the  average  results  of  a  large 
number  of  field  tests  with  U.S.  Army  equipment  (AN-FGC  54) 
employing  quaternary  differentially  coherent  MPSK  under 
flat  fading  conditions  (Bello  65b).  The  equipment  operates 
with  20  subchannels  over  a  voice  band.  The  subchannels 
are  spaced  110  Hz  apart  and  the  signals  are  simultaneously 
keyed  and  mutually  orthogonal  (Kineplex  type).  The  total 
bandwidth  occupancy  is  2400  Hz  ;  the  transmission  rate 
is  20  x  150  bps,  giving  the  system  a  bit  density  of  1.25  or 
4-0.96  db.  The  measured  SNR  is  20.5  db,  yielding  a  utility 
of  —19.54  db.  It  can  be  seen  that  this  point  is  about 
10  db  below  the  theoretical  point  (q  =  4)  on  curve  6.  This 
degradation  is  primarily  due  to  atmospheric  noise  and  flat 
fading.  Operational  periods  with  multipath  and  Doppler 
effects  had  been  excluded  from  the  error  counts.  Diversity 
reception  had  been  applied,  although  no  details  could  be 
found  in  the  paper  about  the  particular  diversity  arrangement 
used.  The  paper,  however,  gives  a  valuable  analysis  of 
diversity  operation  for  binary  PSK  systems. 

Point  8  in  fig.  15  reflects  the  average  results  of  field  tests 
of  a  quarternary  DC-MPSK  modem  over  leased  telephone 
lines  of  approximately  250  miles  in  length  (Evans  et  al  61). 
This  modem  operated  with  2500  bps  over  an  effective  channel 
bandwidth  of  1800  Hz.  It  has  a  single  channel  centred  at 
1667  Hz.  It  can  be  seen  that  in  this  case  the  operational 
equipment  is  only  4  db  below  the  theoretical  value  and  that 
the  bit  density  also  comes  closer  to  its  theoretical  goal 
than  in  the  case  of  point  7. 

Line  9  is  from  another  report  about  tests  of  the  Hughes 
Aircraft  Company  (HC  270)  four-phase  modem  over  Army 
carrier  systems  (Tucker  and  Duffy  (53).  The  system  operated 
for  these  tests  with  1200  bps.  The  first  test  was  back  to  back 
with  only  the  filters  in  the  terminal  equipment  restricting  the 
bandwidth  (3000-Hz  noise  bandwidth).  The  equipment 
needed  7.2  db  SNR  to  run  with  10" 3  error  ratio.  This 
corresponds  to  the  left  end  of  the  lino.  The  second  test 
was  over  six  sections  of  carrier  equipment  with  all  repeaters 
switched  in,  but  no  real  linos  connected.  In  such  tests 
noise  may  be  inserted  from  a  noise  generator  if  the  natural 
noise  of  the  equipment  is  not  adequate.  The  paper  does 
not  specify  in  detail  the  kind  of  noise  generation  and  noise 
measurement  used  during  these  tests.  The  purpose  was 
primarily  to  cheek  the  operation  under  phase  distortions 
and  band  limitations  as  they  are  inserted  by  the  carrier 
equipment.  The  results  of  this  second  test  fall  into  the 
centre  part  of  line  9  in  fig.  15.  The  right  end  of  the  line 
corresponds  to  a  test  where  1 2  sections  of  AN /TCC7  carrier 
equipment  were  connected  in  tandem.  The  fact  that  the 
utility  remains  constant  over  all  tests  indicates  that  the  change 
in  SNR  ratio  was  due  only  to  the  bandwidth  decreasing  with 
the  increasing  number  of  sections  in  tandem.  This  is  an 
interesting  demonstration  of  the  advantage  of  utility  curves. 
Comparing  these  results  with  the  theoretical  curve  shows 
that  the  military  carrier  equipment  enforces  an  operation 
at  much  lower  bit  densities  than  those  theoretically  possible 
or  even  those  achieved  in  radio  transmission. 

Lines  10A  and  B  are  the  results  of  tests  performed  over 
German  telephone  facilities  with  controlled  impulse  noise 
intentionally  inserted  (Von  Hanisch  and  Kettel  (53).  The 
rosidts  are  available  only  for  the  range  of  error  ratios  from 
1 0_1  to  10  ".  When  comparing  these  results  with  others, 
it  is  necessary  to  extrapolate  the  lines  upwards  to  about 
—  12  db  and  —15  db,  respectively,  to  estimate  the  utility 
for  e«  -  10->. 

Point  1 1  is  reported  by  Morioka  et  al  (60).  It.  resulted  from 
tests  of  an  ootonary  DC-MPSK  system  (AN-GSC4,  Kineplex) 
operating  over  specially  selected  leased  lines  (530  miles  of 
H  44  cable)  at  5400  bps.  The  telephone  circuits  have  a 
nominal  bandwidth  of  4  kHz,  but  3200  Hz  were  used  for  the 
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signals.  This  gives  a  bit  density  of  1.69  or  2.28  db.  The 
systems  needed  a  SNR  of  19.6  db  for  10“ 3  bit  error  ratio. 
This  established  the  utility  at  —17.2  db.  The  authors 
mention  that  “  The  impulse  noise  must  not  be  too  objection¬ 
able,  although  the  operation  of  the  equipment  in  t  he  presence 
of  impulse;  noise  had  indicated  that  it  is  not  as  susceptible 
as  one  might  believe.”  The  equipment  operates  with  six 
subchannels  centred  from  935  Hz  to  3135  Hz  with  440  Hz 
separation.  The  digital  rate  in  each  subchannel  is  300  bps. 

Point  12  in  fig.  15  finally  is  inserted  for  comparison  (Langley 
and  Oliver  65).  It  refers  to  the  previously  mentioned  di¬ 
phase  binary  PSK  system  when  used  for  space  booster 
telemetry  over  a  wide-band  channel  of  400  kHz  bandwidth. 
In  this  particular  version  the  system  operates  in  an  error 
control  feedback  mode,  which  makes  it  practically  error 
free,  but  which  shifted  its  bit  density  to  —5.12  db. 

H3.  Auto- Phase  Reference  MPSK  Systems 

Two  or  more  subclasses  of  APR  systems  may  be  discrimi¬ 
nated  according  to  the  mode  used  to  give  the  receiver  the 
phase  reference  and  to  update  it  continuously.  Of  all  the 
conceivable  means  that  can  be  used  to  achieve  this  goal, 
the  differentially  coherent  (DC)  mode  is  most  frequently 
applied.  Russian  authors  speak  of  relative  phase  telegraphy 
(RPT)  and  Bobrov  ((50)  established  an  intricate  classification 
system  of  the  various  versions  which  can  be  applied  to 
transmit  the  phase  reference  together  with  the  information. 
Basically  there  are  two  groups  of  RPT  systems. 

The  first  group  covers  systems  in  which  the  number  of 
possible  phases  of  the  signal  is  equal  to  the  number  of  phase 
differences  used  for  transmission.  The  minimum  phase 
shift  is  zero,  meaning  that  one  of  the  waveforms  is  exactly 
repeated  with  the  same  phase  state  to  express  one  of  the 
characters.  I  This  first  group  includes  the  binary  PSK 
system  with  0  to  180  degrees,  the  ternary  PSK  system  with 
0  to  120  to  240  degrees,  and  the  standard  DC-MPSIv  system 
with  0  to  90  to  180  to  270  degrees.  The  disadvantage  of 
systems  of  this  group  is  the  unchanged  phase  state  which 
results  for  certain  messages.  This  may  cause  difficulties 
in  circuits  and  may  present  problems  for  the  synchronization 
,  extractor  (recovery  of  the  clocking  rate). 

A  special  encoding  mode,  used  in  a  second  group  of  RPT 
systems,  and  called  the  rotating  phase  mode  (Nazarov  64) 
avoids  this !  difficulty  by  providing  two  interlaced  sets  of 
waveforms  offset  in  phase  against  each  other  for  w/q  and 
using  them  alternately.  In  other  words,  the  transmitter 
will  shift  in  a  binary  system  for  90  to  270  degrees  ;  in  a  ternary 
j  system,  for  60  to  180  to  300  degrees  ;  and,  in  a  quaternary 
system,  for  45  to  135  to  225  to  315  degrees.  The  last  mode 
is  the  normal  operation  of  the  Kineplex  system  (Icenbice  57). 

Stein  (64;  performed  a  unified  analysis  of  binary  com¬ 
munications  systems,  arriving  at  a  fundamental  mathe¬ 
matical  formulation  that  should  be  of  importance  to  further 
research  int'o  the  noisy  reference  problem  of  APR-MPSK 
systems.  The  next  step  in  the  development  of  APR  methods 
j  may  well  be  the  inclusion  of  more  than  just  the  last  received 
signal  into  the  process  of  recovering  the  phase  references. 
Good  guidance  for  such  developments  may  be  derived  from 
Proakis  et  al  (64). 

There  are  not  many  MPSK  terminals  in  existence.  There¬ 
fore,  we  prefer  to  list  them  by  developers  and  mention  their 
characteristic  features. 

Tho  Kinaplex  system ,  originally  called  predicted  wave 
signalling  (PWS)  system,  has  already  been  mentioned 
several  times  (Doelz  et  al  57,  Morioka  et  al  60,  Icenbice  57). 
It  operates  with  two,  four,  or  eight  phases  and  prefers  the 
rotational  phase  encoding  mode.  In  its  standard  version 
it  operates  with  many  subchannels  (up  to  20),  which  are  all 
keyed  simultaneously  and  whose  subcarrier  frequencies 
are  so  assigned  that  there  is  a  minimum  of  intersymbol 
I  interference  for  a  given  elementary  length  of  tho  rectangular 
j  signals  (Hosier  57).  Gated  reasonators  (infinite  Q-filters) 


exactly  tuned  to  tho  subcarrier  frequency  act  as  matched 
filters  and  as  phase  storage  devices  at  the  receiving  side. 
Two  sets  of  reasonators  are  alternatively  used  and  quenched 
after  two  elementary  intervals.  They  are  loaded  during  the 
first  interval  of  each  circle  and  they  hold  the  phase  reference 
during  the  second  interval.  Kineplex  has  now  been  in 
development  and  operation  for  more  than  10  years.  Many 
evaluations  have  been  performed  in  comparison  with  other 
systems  in  laboratory  tests  (Easterling  et  al  61),  over  tele¬ 
phone  lines  (Fontaine  and  Gallager  61,  Fontaine  63)  and  over 
radio  channels  (Greim  et  al  65).  Tho  result  may  be  sum¬ 
marized  by  the  statement  that  DC-MPSIv  systems  have 
absolutely  fulfilled  tho  expectations  when  tested  under  the 
conditions  assumed  by  theory,  i.e.,  Gaussian  noise,  linear, 
and  time-constant  channels.  But  when  subjected  to  real 
life  operational  conditions,  the  results  show  degradation  of 
10  db  and  more  against  the  unrealistically  ideal  mathematical 
models. 

One  four-phase  data  modem  of  Hughes  Aircraft  Company 
(HC  270)  operates  over  a  single  subcarrier  frequency  located 
at  the  centre  of  a  voice  band  (Evans  et  al  61).  It  is,  therefore, 
less  costly  than  the  Kineplex  terminals,  but  is  using  the 
available  bandwidth  less  efficiently  (no  orthogonal  multi¬ 
signal  composite  waveforms)  and  is  more  subject  to  phase 
distortions,  since  the  spectrum  of  each  signal  ranges  over  the 
whole  voice  band.  It  can  operate  with  a  rate  of  600,  1200, 
or  2400  bps  over  telephone  lines  of  the  switched  network 
and  with  4800  bps  over  higher  quality  voice  circuits.  It 
applies  an  auto-correlation  technique  to  identify  the  change 
in  phase  (Toffler  and  Buterbaugh  61),  and  it  incorporates 
tho  design  of  a  phase-locked  loop  (PEL),  and  an  advance- 
retard  counter  to  recover  timing  information  from  the 
received  signal  (Buterbaugh  62). 

Another  four-phase  data  modem  of  Hughes  Aircraft 
Company  (IiC  273)  was  specifically  designed  for  high  fre¬ 
quency  (i.e.,  short  wave)  radio  transmission  and  operates 
with  12  subchannels,  each  using  a  digit  length  of  10  ms 
(200  bps)  so  that  the  total  rate  is  2400  bps.  The  digital 
length  of  10  ms  is  sufficient  to  obtain  satisfactory  long- 
haul  and  short-haul  operation  over  a  wide  range  of  frequencies. 

A  special  differentially  coherent  quadriphase  modem 
developed  by  Republican  Aviation  Corporation  operates  on  a 
single  carrier  over  a  wideband  microwave  channel  at  13.5  GHz 
(Fisch  63).  It  was  used  to  demonstrate  the  transmission  of 
digitized  television  signals  digitally  encoded  at  24  megabits 
per  second  (mbps).  The  quadriphase  modulation  is  straight¬ 
forward  in  the  phases  0,  90,  180  and  360  degrees.  The 
inclusion  of  0-degreo  phase  does  not  pose  any  problem  in 
this  easo  (megabit  rates  and  TV- PCM). 

The  special  case  of  q  =  4  (quadriphase)  of  MPSK  systems 
has  been  frequently  used  as  a  multiplex  system  to  transmit 
two  independent  binary  information  flows  simultaneously. 
Indeed,  the  original  idea  of  Kineplex  resulted  from  this 
requirement  (Doelz  et  al  57).  It  has  been  revived  for  the 
transmission  of  two  independent  PCM  channels  (Anderson  63). 
The  same  idea  leads  also  to  the  simultaneous  transmission 
of  two  independent  binary  information  streams  via  the  two 
quadrature  components  of  a  subcarrier  modulated  in  DSB 
(O’Neill  and  Saltzberg  66).  If  both  channels  are  exactly 
synchronized  and  of  equal  amplitude  and  if  exactly  rectan¬ 
gular  waveforms  are  used,  the  resultant  signals  are  identical 
for  DSB-MUX  and  for  MPSK  q  =  4,  the  first  group  of 
encoding  modes. 

A  critical  circuit  problem  in  MPSK  receivers  is  the  question 
of  a  limiter  in  the  intermediate  frequency  (IF)  amplifier 
versus  a  linear  amplifier  (Zabronsky  61).  There  is  apparently 
no  completely  satisfying  answer  to  this  question.  It  depends 
on  the  channel,  the  type  of  information,  tho  type  of  coding, 
etc.  The  limiter  question  has  to  be  decided  in  the  context 
of  an  integrated  system. 

Thus  far  wo  have  discussed  quaternary  and  octonary 
MPSK  systems,  yet  theory  showed  t  hat  the  highest  utility 
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is  for  q  =  3.  Still  we  could  not  find  a  report  in  the  literature 
about  an  actually  developed  ternary  system,  besides  the 
multi-lock  system  of  the  Sobertshaw- Fulton  Company  (Crafts 
58),  which  used  the  third-phase  state  as  a  means  to  enforce 
sufficient  spectral  energy  at  the  carrier  frequency  and  not  to 
carry  information. 

H4.  Separate  Phase  Reference  MPSK  Systems 

In  the  introduction  to  this  section  we  divided  the  MPSK 
systems  roughly  into  two  large  groups  of  systems  :  auto¬ 
matic  phase  reference  (APR)  systems,  discussed  in  sub¬ 
section  H3,  and  separate  phase  reference  (SPR)  systems, 
to  be  discussed  here.  The  SPR  systems  arc  the  counterpart 
to  the  very  successful  MASK-SSB  systems  with  pilot  carrier(s) 
which  we  discussed  in  subsection  Fo.  Yet  in  the  MPSK 
case,  until  recently,  much  less  attention  has  been  given  to 
systems  with  separate  reference  channel  than  to  the  previously 
discussed  systems  with  automatic  phase  reference.  Bussgang 
and  Leiter  (G5a,  60)  recently  reviewed  the  analytical  back¬ 
ground  of  SPR-MPSK  systems  and  referred  to  two  different 
approaches  for  the  transmission  of  the  phase  reference  inde¬ 
pendently  of  the  information. 

The  first  approach  uses  one  (or  more)  reference  tone  of 
constant  frequency.  Such  special  reference  subchannels 
can  be  operated  with  a  very  small  bandwidth,  thus  permitting 
the  extraction  of  their  centre  frequency  in  the  receiver  with 
filters  of  phase-lock  loops  of  very  small  noise  bandwidth. 
The  phase  of  this  reference  signal  must  be  adjusted  in  tho 
receiver  to  compensate  for  the  frequency  difference  between 
the  information  carrier  and  the  reference  carrier  and  also 
for  the  different  phase  delay  due  to  the  different  bandwidth 
in  tho  two  channels.  The  problem  of  finding  the  optimum 
power  division  between  the  information  channel(s)  and  the 
reference  channel (s)  is  critical  (Stiffier  64b).  A  system  which 
applies  this  first  approach  is  the  DEFT  system  developed  by 
the  Electronics  Division  of  General  Dynamics  Corporation  in 
Rochester,  New  York  (DeHaas  64.  65).  This  is  a  multi¬ 
signal  modem  with  many  subchannels  operating  in  frequency 
division  in  a  voice  frequency  band  of  from  300  to  3000  Hz. 
Many  reference  frequencies  are  spread  across  the  band, 
spaced  150  Hz  apart.  The  information  subcarriers  are 
modulated  in  quaternary  MPSK  with  an  elementary  length 
of  26  2/3  ms.  Tho  total  modem  is  designed  for  2400  or 
3600  bps  and  it  has  been  tested  over  a  7000-mile  ionospheric 
path.  Another  system  applying  the  reference  tone  approach 
is  tho  phase  telemetry  comparison  system  used  by  the  U.S. 
Naval  Space  Surveillance  System  (Easton  and  Downey 
66).  It  uses  24  subchannels  of  40  Hz  bandwidth  spaced 
100  Hz  apart.  Only  two  reference  tones  are  needed,  since 
the  system  operates  over  telephone  lines.  The  two-reference 
frequencies  are  placed  at  1000  Hz  and  1100  Hz.  Their  beat 
frequency  is  the  100-Hz  timing  and  phase  reference. 

The  second  approach  for  the  transmission  of  the  phase 
reference  uses  phase  multiplexing  within  a  subchannel  to 
transmit  the  phase  reference.  One  quadrature  component 
of  the  sine-subcarrier  carries  binary  PSlv  signals,  while 
the  other  phase  quadrature  component  remains  essentially 
unkeyed  to  serve  as  a  phase  reference.  The  Kathryn  modem 
developed  by  General  Astrionics  Corporation  (Bello  65a) 
shows  an  application  of  this  approach.  This  modem  modu¬ 
lates  a  largo  number  of  subcarriers  in  phase  quadrature, 
whereby  one  component  is  modulated  by  the  information 
in  binary  PSK  while  the  other  phase  quadrature  component 
is  modulated  by  a  binary  pseudo  noise  (PN)  sequence  to 
serve  as  an  accurate  phase-and-timing  reference.  All 
subcarriers  carry  their  own  reference  in  this  way.  The 
quadricorrelator  (Hannan  and  Olson  61)  is  an  interesting 
variety  of  phase-lock  loops  which  employs  two  phase  detectors 
commonly  known  as  the  I  (or  in-phase)  detector  and  the  Q 
(or  quadrature  phase)  detector.  It  may  bo  useful  in  systems 
with  quadrature  reference  channels. 

A  closely  related  idea  was  expressed  by  Rensliaw  (65a), 


who  suggested  the  use  (at  specified  time  instants)  of  the  phase 
difference  between  two  subcarriers  of  different,  but  constant 
and  well-related,  frequencies  to  express  the  information 
and/or  any  phase  reference. 

The  application  of  limiters  in  tho  receiver  subchannels 
for  either  the  information  carrier  or  the  reference  tones  is 
discussed  by  Zabronsky  (61). 

H5.  Applications  of  MPSK  Systems 

Much  of  what  has  been  said  in  the  previous  subsections 
already  indicated  that  MPSK  systems,  like  MFSK  systems, 
find  wide  applications,  primarily  in  time  variable  channels. 
The  key  problem  is  again  the  question  of  how  to  maintain 
accurate  phase  references  over  such  time  variable  channels 
(Bussgang  and  Leiter  65b). 

The  statistical  characteristics  of  fading  channels  was 
extensively  explored  in  the  past ;  some  of  the  important 
publications  were  reviewed  in  Part  I,  sections  B2  and  D2. 
Lutz  et  al  (59),  for  example,  experimentally  explored  prior 
to  1959,  the  phase  distortions  suffered  by  pulses  of  1  ms  and 
20  /is  length  when  passing  ionosphere  paths  of  3000  km-length 
and  beyond.  They  describe  a  data  transmission  system  with 
phase  change  signalling  that  operated  with  millisecond  pulses. 
Based  on  such  knowledge  it  was  possible  for  many  research 
teams  to  lay  out  further  statistical  tests  with  the  classical 
data  transmission  systems  when  operating  under  fading 
conditions  and  to  attempt  the  design  of  more  sophisticated 
systems  with  special  antifading  or  antimultipath  characteris¬ 
tics. 

Glenn  and  Liebermann  (62)  experimentally  analyzed  the 
performance  of  DC-PSK  in  comparison  with  DSB  quadrature 
operation  and  noncoherent  FSK  under  fading  and  jamming 
conditions.  Their  results  showed  that  “  tho  DC-PSK  system 
was  most  critical  to  fast  fading,  while  tire  noncoherent 
FSK  was  somewhat  better  than  the  quadrature  detection 
system  ”.  They  also  recognized  that  “  the  shape  of  the 
spectral  density  of  the  signal  fading  can  have  a  significant 
effect  on  error  probability  performance  ”.  Fontaine  (63) 
reported  on  comparisons  between  Kineplex  and  other  data 
systems  over  telephone  circuits  carried  via  forward  propaga¬ 
tion  tropospheric  scatter  circuits.  The  transmission  of  the 
modified  diphase  signals  (section  HI)  over  radio  links  was 
reported  by  Kolarck  and  Paramithas  (63),  and  Ferrell  and 
Matava  (63)  related  their  experience  with  a  point-to-point 
DC-PSK  system  operating  over  MF  radio  (2  to  3  MHz). 

The  experience  gained  from  the  above-mentioned  experi¬ 
ments  and  from  many  other  observations  leads  to  a  number 
of  analytical  approaches  and  to  some  design  eff  orts,  all  aiming 
at  a  solution  of  the  crucial  problem  of  getting  digital  data 
over  a  fading  channel. 

Lindsey  (64)  published  a  theoretical  analysis  of  channels 
with  “  resolvable  multipath  and  diversity  reception  ”,  for 
which  he  borrowed  the  term  “  multichannel  ”  from  an  earlier 
paper  of  R.  Price  (62).  Another  theoretical  analysis  of 
Becker  et  al  (64)  was  previously  mentioned  in  this  survey. 
The  paper  also  contains  an  analysis  of  the  transmission  of 
DC-PSK  signals  over  a  fading  tropospheric  scatter  circuit, 
which  may  be  usefid  for  further  MPSK  developments. 
Gerastovskiy  (65)  contributed  an  investigation  of  the  noise 
immunity  of  phase  telegraphy  under  Rayleigh  fading 
conditions. 

On  the  practical  side,  attempting  the  design  of  MPSK 
systems  operating  satisfactorily  under  fading  conditions,  wo 
have  previously  referred  to  the  DEFT  design  (Walker  64,  65, 
DeHaas  65,  Greim  et  al  65)  and  to  the  Kathryn  modem  (Bello 
65a,  Bussgang  and  Leiter  66).  From  all  these  reports  in 
the  open  literature  this  reviewer  can  reach  only  one  con¬ 
clusion  :  the  problem  of  data  transmission  over  fading 
(multipath)  media  is  not  yet  solved.  But  much  credit  should 
he  given  to  the  research  and  development  workers  who  have 
contributed  to  the  understanding  of  this  crucial  problem. 
The  result  of  all  their  efforts  is,  at  least,  a  much  better  under- 
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.standing  of  the  complex  natural  effects  that  we  are  trying 
to  master  ;  at  most,  the  hope  that  we  have  already  found 
the  best  design  approach  and  that  it  is  merely  a  matter  of 
further  small  improvements  to  solve  completely  the  problem 
of  data  transmission  over  fading  channels. 

It  cannot  be  ascertained  at  this  time  if  MPSK  will  be 
superior  to  other  MXSK  systems  under  all  circumstances. 


The  corresponding  problem  in  analog  transmission  has  never 
foimd  a  unique  answer  to  the  problem  of  which  modulation 
method  is  the  best.  We  still  have  AM,  DSB-SC,  SSB, 
VSB,  FM,  PhM  and  hybrid  systems  side  by  side.  This 
reviewer  believes  the  same  will  be  the  case  with  digital  trans¬ 
mission  modes.  He  further  believes  that  binary  systems 
in  general,  due  to  their  simplicity,  will  continue  to  be  used 
concurrently  with  nonbinary  systems. 


Part  III — Coded  Multidimensional  Systems 


Introduction 

Part  III  deals  with  systems  using  a  group  (or  a  sequence) 
of  digits  as  ono  unit,  called  a  word.  Each  digit  may  assume 
one-out-of-q  states  of  a  well-defined  waveform.  Alternately 
during  each  digital  interval  the  transmitter  may  select 
one-and-only  one  out  of  an  alphabet  of  q  different,  but 
well-defined,  waveforms.  The  systems  discussed  in  Part  III 
follow  in  the  logical  order  of  a  multidimensional  communica¬ 
tions  system.  This  order  is  explained  in  section  1.  Accord¬ 
ing  to  the  order,  section  J  deals  with  codebooks,  and  with  the 
receiving  techniques.  Section  Iv  discusses  mathematical 
models  of  these  monosignal  multidimensional  data  trans¬ 
mission  systems  and  section  L  reviews  the  experimental 
results  that,  have  been  published  so  far. 

Finally,  Part  IV  is  devoted  to  the  waveforms  that  may  be 
used  in  the  implementation  of  monosignal  nonbinary  systems 
over  bandlimited  channels. 

Some  systems,  which  cannot  be  discussed  in  this  paper, 
may  bo  culled  polysignal  systems  because  the  restraint 
imposed  in  our  present  discussion,  that  in  each  time  interval 
one-and-only-one  waveform  must  be  transmitted,  is  no 
longer  in  effect.  These  polysignal  systems  use  the  super¬ 
position  (the  simultaneous  transmission)  of  many  signals 
to  achieve  a  higher  information  density.  They  form  a 
rather  complex  “  composite  signal  ”.  Many  of  the  basic 
systems  of  Parts  II  anil  III  are  used  in  the  polysignal  systems 
on  a  subsystem  basis  in  subchannels  (frequency  division) 
or  in  more  general  subspaces  of  the  total  signal  space  of  the 
composite  signal. 

I.  The  General  Multidimensional  Transmission  Mode 

Part  II  of  this  paper  presented  the  fundamentals  and  the 
state-of-the-art  of  ono-dimensional  nonbinary  systems  which 
we  also  called  multistate  systems.  The  definition  of  these 
systems  indicated  that  a  single  parameter  of  a  given  wave¬ 
form  (ideally  a  rectangular  pulse)  was  permitted  to  assume  q 
different  states.  Thus  the  system  offered  a  single  degree  of 
freedom,  which  is  the  reason  for  calling  the  systems  described 
in  Part  II  one-dimensional  systems.  It  is  evident,  for  example, 
that  a  single  signal  (pulse)  cannot  assume  two  different  ampli¬ 
tude  levels  at  the  same  time  :  their  superposition  would 
yield  only  a  third  anil  different  amplitude  level  and  the 
receiver  would  be  in  no  position  to  recover  the  two  original 
amplitude  levels.  This  apparently  is  different  when  the 
two  original  amplitude  levels  are  used  to  modulate  two 
different,  independent  parameters  of  a  more  complex  wave¬ 
form.  We  speak  in  this  case  of  a  two-dimensional  signal. 
Extending  this  idea  to  n-dimensions  is  the  subject  of  Part  II 1 . 

Whether  one  uses  for  the  design  of  a  multidimensional 
system  sequences  of  a  single  type  of  waveform  (Part  III) 
or  selections  from  an  alphabet  of  band-limited  waveforms 
(Part  IV),  one  must  apply  rather  complex  receiving  techniques 
to  recover  the  information  from  the  multidimensional  signal. 
These  techniques  involve  filtering  or  correlation  devices, 
decoding  circuits,  and  decision  methods.  These  receiving 
I  techniques  are  discussed  in  the  secontl  part  of  section  J. 


The  other  sections  of  Part  III,  sections  K  and  L,  describe 
complete  multidimensional  systems.  Their  mathematical 
models  are  discussed  in  section  K  ;  and  their  operational 
performance,  in  section  L. 

With  monosignal  operation  in  mind  as  the  only  restriction 
imposed  on  the  multidimensional  systems  of  Part  III,  we 
may  now  turn  to  fig.  16  to  see  the  essential  elements  which  a 
typical  multidimensional  system  requires.  Wo  notice  that  a 
multidimensional  system  needs  a  codebook  and  a  waveform 
library  for  its  specification.  The  codebook  (discussed  in 
Part  III)  will  be  programmed  into  the  transmitter  encoder 
and  into  the  receiver  decoder.  The  waveform  library 
(discussed  in  Part  IV)  will  bo  programmed  into  the  trans¬ 
mitter  modulator  matrix  and  into  the  receiver  demodulator 
matrix. 

Figure  16  shows  tho  information  flow  beginning  at  the 
upper  left  side  with  a  binary  input.  The  serial-to-parallel 
converter  takes  a  block  of  n  bits  into  the  encoder,  where  this 
input  message  is  translated  into  the  selection  of  one  of  the  M* 
codebook  words.  Apparently,  the  minimum  value  for  M 
must  be  2",  so  that  each  of  the  possible  input  messages  may 
get  a  code  word  assigned.  Conversely,  each  code  word, 
when  identified  by  the  decoder  in  the  receiver,  will  uniquely 
correspond  to  one  binary  output  message  of  n  bits.  It  is, 
however,  possible  to  make  M  larger  than  2"  and  to  assign 
to  some  or  to  all  the  input  messages  not  ono  but  several 
code  words.  Some  other  code  words  may  be  service  words 
not  assigned  to  any  input  message.  The  reasons  for  such 
redundant  designs  may  be  the  need  for  special  synchroniza¬ 
tion  words,  privacy  of  transmission,  addressing  of  special 
receivers,  and  so  forth. 

The  codebook  characters  will  preferably  be  selected  from  an 
alphabet  of  the  same  size  ns  the  waveform  library.  Evidently, 
the  codebook  may,  in  its  simplest  form,  consist  of  a  single 
row.  This  is  a  special  case  of  a  binary  system  with  on-off 
keying  of  a  rather  complex  but  unique  waveform  deter¬ 
mined  by  this  single  word  of  the  codebook.  The  codebook 
characters  may  appear  inside  the  equipment  as  multilevel 
pulses,  as  binary  groups,  or  simply  as  the  activation  of  different 
wires.  Whatever  the  design  may  be,  the  effect  must  be  that 
character  A  causes  the  selection  of  waveform  A  from  the 
waveform  library  ;  character  B,  the  selection  of  waveform  B  ; 
anil  so  on  to  character  and  waveform  Q,  which  is  assumed 
to  bo  the  last  letter  in  the  waveform  alphabet  (fig.  16). 

Many  multidimensional  systems  operate  on  the  basis 
that  the  length  of  a  coilebook  word  corresponds  to  the 
number  of  timeslots  (subintervals)  that  are  provided  for 
one  transmission  word  (also  called  a  transmission  sequence). 
Again,  exceptions  are  possible  anil  synchronizing  or  address¬ 
ing  slots  may  be  added  in  the  modular  matrix.  The  encoder 
will  select  one  codebook  word  for  each  input  word  and  forward 
it  to  the  hold  circuit  to  have  it  ready  for  translation  into  a 
sequence  of  transmission  waveforms.  This  hold  circuit 

*In  Part  I  wo  used  “  m  ”  for  the  number  of  digits  (states  or 
levels)  iu  a  one-dimensional  nonbinary  system.  Wo  now  uso  “  M 
for  the  number  of  words  in  a  multidimensional  codobook. 
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Figure  16. — Codebook  and  waveform  library  in  multidimensional 
nonbinary  systems. 


may  be  necessary  because  either  the  encoder  or  the  modu¬ 
lation  matrix,  or  both,  may  need  the  complete  message 
before  they  can  begin  their  respective  translations.  The 
modulator  matrix  will  select  the  correct  waveform  for  each 
character  held  in  the  hold  circuit  and  transmit  the  final 
sequence  in  time  division  into  the  channel.  Several  charac¬ 
ters  could  be  transmitted  in  frequency  division  (i.e.,  simul¬ 
taneously)  or  with  overlapping  waveforms,  but  such  systems 
fall  under  the  concept  of  polysignal  systems  or  multiplex 
systems  and  will  not  bo  discussed  in  the  present  paper. 

Figure  10  shows  that  a  codebook  word  will  consist  of  I  q-ary 
characters.  The  following  relationship  must  hold  to  avoid 
any  loss  of  information  : 

q'  >  M  >  2"  (47) 

If  the  equality  sign  is  applicable  on  the  left  side  of  the 
equation,  the  codebook  is  complete.  This  means  that  all 
possible  code  words  that  can  bo  formed  with  q-ary  characters 
in  1  positions  have  been  used  as  transmission  code  words. 
Such  designs  will  be  attempted  when  bandwidth  is  at  a 
premium  and  the  noise  power  in  the  channel  is  low.  Systems 
of  this  kind  will  operate  at  a  high  bit  density  when  selecting  a 
highly  bandlimited  waveform  library;  i.e.,  they  will  operate 
on  the  right  side  of  the  utility  chart  (figs.  1  and  2  of  Part  1). 
On  the  other  hand,  a  designer  can  made  q  and  1  so  large  that 
only  a  small  number  of  all  possiblo  codebook  words  will  be 
used.  Such  a  redundant  design  will  require  more  than  the 
minimum  bandwidth  for  the  transmission  channel.  Indeed, 
many  redundant  nonbinary  systems  require  more  than  the 
bandwidth  of  a  nonredundant  binary  system,  in  which  case 
they  will  operate  on  the  loft  side  of  the  utility  chart.  Tho 
advantage  of  the  nonbinary  mode  of  operation  (as  discussed 
in  Part  I)  is  higher  utility. 

In  all  cases  where  M  (the  number  of  codobook  words)  is 
much  smaller  than  q1,  a  major  problem  is  the  selection  of  the 


best  subset  of  M  codebook  words  from  the  complete  set  of  all 
possible  words.  This  is  the  same  problem  as  the  selection 
of  error-correcting  block  codes.  Much  mathematical  material 
is  available  in  error-correction  coding  theory.  Further 
details  are  given  later  in  Part  III. 

The  specification  of  a  waveform  library  for  tho  special 
case  1  =  1  (one-dimensional  systems)  has  already  been  dis- 
discussed  in  Part  II.  There,  it  was  assumed  that  all  q 
waveforms  differ  only  in  one  dimension ;  we  called  that 
case  one-dimensional  multistate  systems.  Special  problems 
of  waveform  selection  have  also  been  discussed  in  Part  I 
in  subsections  B1  and  Dl.  From  these  reviews  we  may 
recall  that  the  waveform  library  may  be  designed  for  high 
bit  density,  for  medium  bit  density,  or  for  low  bit.  density. 
In  the  first  case  the  product  Bc  =  BTC  will  be  made  as  small 
as  possible  (B  is  the  essential  bandwidth  occupancy;  T0  is 
the  duration  of  a  waveform).  The  smallest  theoretical 
case  is  Bc  =  0.5,  the  so-called  Nyquist  rate.  Tho  designer 
will  try  to  make  q  larger  than  2BTC  ,  the  maximum  number 
of  orthogonal  waveforms.  This  is  the  rule  in  MASK  and 
MPSK.  In  tho  second  case  (medium  bit  density)  q  will 
be  close  to  j3r  ,  the  signal  base  of  one  character  of  the  wave¬ 
form  library.  Most  designers  will  prefer  to  use  orthogonal 
sets  of  waveforms.  This  is  the  case  in  MFSK  and  in  special 
systems  with  closed  orthogonal  sets  of  waveforms.  In  the 
third  caso  (low  bit  density),  sets  of  waveforms  with  minimal 
cross-correlation  coefficient  will  be  preferred. 

Following  the  information  flow  in  fig.  16  through  the 
channel,  we  first  notice  that  the  signals  leaving  the  trans¬ 
mitter  are  grouped  into  words  of  length  T,0  ,  each  consisting 
of  1  slots  (or  subintervals)  of  length  Tc  .  Notice  that  in 
fig.  16  the  words  are  always  grouped  like  a  train  in  tho 
direction  of  the  arrows  ;  i.e.,  in  the  upper  row  the  first  time 
slot,  T,  ,  is  at  tho  right  side  ;  in  the  lower  row,  at  tho  left 
side. 
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On  entering  the  receiver  each  transmission  word  is  passed 
j  through  the  analog  signal  processor  (Part  I,  C3  and  D3). 
In  the  demodulator  matrix  each  timeslot  is  synchronously 
correlated  with  all  waveforms  of  the  full  library ;  the  result 
ol  this  operation  is  entered  into  the  cross-correlation  matrix 
where  each  row  corresponds  to  one  of  the  transmission  slots 
j  and  each  column  corresponds  to  one  of  the  q  waveforms. 

1°  illustrate  this  process  we  assume  that  the  first  codebook 
j  word  has  bfeen  transmitted.  The  first  slot,  T,  ,  contained 
the  waveform  character,  F  ;  the  second  slot,  N,  followed  by 
■  Q  and  B  until  the  1th  character  was  transmitted  as  wave¬ 
form  C.  In  a  noise-free  and  distortion-free  test  transmission, 
the  cross-correlation  matrix  will  have  in  each  row  only  ono 
j  very  high  positive  value  (autocorrelation  coefficient,  norma¬ 
lized  to  1)  and  all  other  values  will  be  much  smaller  (corres¬ 
ponding  to  the  normalized  cross-correlation  coefficients). 
In  the  first  row  the  1  should  be  in  the  F  column  ;  in  the  second 
row,  in  the  X  column  ;  and  so  on  until  the  1th  row,  which 
,  should  have!  a  “  1  ”  in  the  C  column.  In  an  actual  trans¬ 
mission,  noise  and  distortions  will  modify  this  ideal  situation. 
This  is  indicated  in  fig.  1(1  by  postulating  that  the  highest 
value  in  the  first  row  would  be  in  the  F  column  (not  shown) ; 
the  highest  value  in  the  second  row  would  appear  in  the  A 
column  (marked  l) ;  and,  in  the  1th  row,  in  the  Q  column. 
Some  systonjs  will  make,  on  the  basis  of  the  maximum  value, 
a  (imractor-hy -character  decision  right  at  this  place  or  even 
inside  the  demodulator  matrix.  In  this  case  they  generate 
an  error  in  the  second  row  by  deciding  that  A  has  been 
received  instead  of  N,  and  another  error  in  the  last  row. 
Those  errors  may  or  may  not  be  corrected  later.  More 
sophisticated  systems  will  try  to  save  the  runner-up  informa¬ 
tion,  and  ca -ry  on  to  the  decoder  at  least  the  information 
about  the  second  anil  third  highest  cross-correlation,  leaving 
the  final  dec  ision  to  the  decoder.  The  most  sophisticated 
systems  will  forward  the  complete  cross-correlation  matrix 
with  all  cross-correlation  coefficients  (Floyd  and  Nuttall  65). 
In  each  case  the  decoder  will  perform  a  second  cross -correla¬ 
tion  operation.  In  the  simplest  ense  it  will  t  ake  the  tentatively 
decided  word  in  column  B'  (in  our  case,  the  word  F'A'Q'B' 
j.  ■  •  Q'),  correlate  it  with  all  M  codebook  words,  and  select 
that  word  which  differs  in  the  least  number  of  positions. 
In  a  more  cjomplox,  but  also  more  efficient,  receiver,  the 
decoder  will  receiver  the  actual  values  of  the  cross-correlation 
coefficients  of  the  winners  in  each  row  and  use  them  for  the 
calculation  of  t  he  distance  (section  D1  in  Part  I)  between  the 
received  noisy  word  and  the  closest  clean  words  of  the  code¬ 
book  that  is  i  stored  in  the  decoder.  In  the  most  complex 
receiving  system  the  whole  cross-correlation  matrix  will  be 
used  by  the  decoder,  which  will  reconstruct  the  noisy  version 
Of  each  possibly  received  word  by  using  the  actual  cross¬ 
correlation  coefficients  of  each  character  in  each  word. 
The  decoder  will  then  attempt  to  find  the  noisy  word  that  has 
the  smallest  distance  from  any  one  of  the  M  codebook  words. 
}t  will  decide  that  the  word  with  the  smallest  such  distance 
has  been  received. 

The  word  the  decoder  has  finally  calculated  ns  the  most 
likely  received  word  will  be  translated  back  into  its  binary 
equivalent  with  the  help  of  the  codebook.  This  is  symbolized 
by  the  %'  column  at  the  left  lower  end  of  fig.  16.  If  the 
combined  demodulator,  decider/decoder  process  selected  the 
wrong  codebook  word,  there  must  be  ono  or  more  errors 
in  the  binary  output  word  when  compared  with  the  binary 
input  word.  The  ratio  of  such  errors  to  the  total  number 
Of  transmitted  information  elements  (bits)  is  the  final  quality 
criterion  of  the  system  (section  Cl  in  Part  I).  Apparently 
the  number  ol’  final  output  errors  will  have  something  to  do 
with  the  translation  rules  which  are  applied  at  the  transmitter 
encoder  when  going  from  binary  input  words  to  codebook 
words.  If  neighbouring  codebook  words  (words  with  small 
distance)  correspond  to  neighbouring  words,  the  total  error 
probability  can  be  minimized.  This,  and  a  number  of  other 
gleneral  problems  of  multi -dimensional  systems,  are  discussed 
iiji  the  next  three  sect  ions  in  the  form  of  a  review  of  the  present 
research  in  this  area. 

I 
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J.  The  Encoding  and  Reception  Process  of  Coded  Multi¬ 
dimensional  Systems 

We  discussed  some  of  the  fundamental  encoding  problems 
in  section  D1  in  Part  I.  Here  wo  intend  to  go  deeper  into 
the  nonbinary  coding  and  reception  processes. 

.11.  Binary  to  M-ary  Translation  Codes 

Tho  translation  in  the  encoder  from  binary  to  M-ary  had 
the  attention  of  many'  research  workers  for  problems  other 
than  nonbinary  information  transmission.  In  connection 
with  pulse  code  communications  systems,  F.  Gray  (53) 
received  a  U.S.  patent  on  the  now-famous  Gray  code  in 
1953.  At  about  the  same  time,  H.  J.  Gray,  Jr.,  Levonian, 
and  Rubinoff  applied  such  codes  to  analog-to-digital  con¬ 
verters  (H.  J.  Gray,  Jr.  et  al  53).  A  year  later  F.  A.  Foss  (54) 
investigated  the  application  of  such  codes  to  digital  control 
sy'stems.  Flores  (56)  summarized  the  results  of  this  particular 
branch  of  coding  theory,  with  the  help  of  elementary  number 
theory',  under  the  title  :  “  Reflected  Number  Systems  ”. 

Since  then  reflected  binary  codes  (as  Gray  codes  are  also 
called)  have  been  the  subject  of  more  exhaustive  research 
by  Patterson  (57),  Gilbert  (58),  and  Okunev  (65,  64).  The 
authors  of  these  later  publications  clearly  had  in  mind  the 
application  to  nonbinary'  systems. 

Gray'  codes,  which  belong  to  the  unit  distance  number 
representation  systems  (Patterson  57),  may  be  considered 
as  cyclic  codes  or  closed  codes  (Okunev  64).  The  term 
“  unit  distance  ”  indicates  that  each  code  word  differs  only 
in  one  position  from  the  two  neighbouring  code  words  when 
they  are  assigned  fixed  positions  in  a  codebook.  The  example 
of  a  triple  Gray  code  in  fig.  17  makes  this  clear.  The  eight 
possible  binary  code  words  with  three  elements  are  arranged 
on  the  left  side  in  the  order  W,  to  W8  .  For  example, 
the  code  word  Ws  differs  in  only  one  position  from  the  code 
words  W,|  and  W6  next  to  it  in  the  codebook.  The  right 
side  of  fig.  17  shows  the  cyclic  or  closed  character  of  these 
codes  which  gives  the  first  word  W,  and  the  last  word  Ws 
the  same  advantage  as  all  the  other  words.  They  are  con¬ 
sidered  neighbouring  words  and  as  such  differ  in  only'  one 
position. 

These  codes  are  of  importance  for  nonbinary  systems  when 
selecting  the  translation  mode  from  the  binary  input  to  the 
eodebook  of  M  q-ary  message  sequences  of  length  1.  The 
one-dimensional  systems  discussed  in  Part  II  especially 
demonstrate  the  interconnection  between  tho  class  of  wave¬ 
forms  and  t  he  encoding  procedure.  These  systems  operate, 
as  explained,  on  a  digit-by-digit  basis.  The  codebook 
degenerates  to  a  single  column  (1  =  l).  In  this  case  it  is 
particularly  important  that  t  he  most  likely  occurring  demod¬ 
ulator  errors  cause  the  least  number  of  residual  binary 
errors  in  the  output  words.  Gilbert  (58)  investigated  codes 
with  a  total  number  of  codo  words  (M)  which  is  not  a  power 
of  “  2  ”,  For  example,  codebooks  for  M  =  10  are  very 
important.  Notice,  however,  that  reflected  binary  codes 
have  no  particular  advantage  in  one-dimensional  sy'stems 
if  there  is  no  inherent  unsymmetry  in  the  q-ary  transmission 
error  probabilities  of  the  various  states.  In  certain  PCM 
systems,  for  example,  where  the  transmission  signals  are 
binary  and  where  all  errors  are  equally  likely,  Gray  codes 
have  no  basic  advantage  over  other  codes  (Filipowsky 
and  Muehldorf  65a,  Chapter  7,  p.  575).  Yet,  they'  may  be 
of  importance  in  multidimensional  nonbinary  systems,  even 
if  all  waveforms  of  the  waveform  library  have  equal  error 
probabilities. 

,12.  The  E rolii lion  of  Block  Codes  ( Codebooks ) 

Tho  research  on  codehooks  has  been  active.  Many  mathe¬ 
matical  models  for  suitable  codebooks  have  been  published 
since  1949  when  Shannon  postidated  the  conditions  which 
designers  must  moot  when  striving  for  high  efficiency  systems. 
We  discussed  these  general  conditions  in  Part  I  (C2  and  Dl). 
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Figure  17. — Examples  of  the  triple  Gray  code. 


Here  we  mention  the  basic  approaches  to  realizable  block 
codas  (codebooks)  and,  in  the  following  sections,  we  discuss  a 
number  of  actually  worked-out  codebooks. 

The  earliest  attempt  to  specify  a  codebook  in  line  witn 
Shannon’s  recommendations  is,  most  likely,  the  randomiza¬ 
tion  approach  published  by  Rice  (50).  He  demonstrates  that 
condition  3  of  Shannon,  the  white  noise  character  of  the 
signals,  can  be  met  by  a  codebook  where  the  letters  for  each 
code  word  are  selected  at  random  from  a  population  of  real 
numbers  which  has  the  same  distribution  as  the  noise.  He 
investigated  a  Gaussian  distribution  and,  in  terms  of  our 
example  in  fig.  16,  he  would  select  the  alphabet  of  q  letters 
so  that  only  a  few  would  have  very  large  positive  or  negative 
values  and  the  majority  would  centre  around  zero,  establish¬ 
ing  a  Gaussian  amplitude  distribution.  From  this  popula¬ 
tion  ho  would  draw  a  letter  (real  number)  for  the  first  field 
(row  1,  col.  1)  of  the  codobook  matrix,  another  real  number 
for  the  second  field  (row  1,  col.  2),  and  so  on,  until  the  first 
word  is  formed.  The  second,  third,  etc.  word  would  be 
formed  in  the  same  way  by  picking  values  from  a  large 
Gaussian  population  at  random  until  tho  codebook  of  M 
words  is  finished.  Naturally,  tho  receiver  must  get  the 
complete  eodebook  in  advance  of  any  transmission.  Rice 
proved  that  a  system  on  this  basis  can  achieve  Shannon’s 
upper  bound  of  the  channel  capacity,  but  only  for  M  ap¬ 
proaching  infinity. 

Gilbert  (52)  approached  the  same  problem  from  the  other 
end,  building  up  codebooks  of  very  small  dimensions  in 
such  a  way  that  all  columns  correspond  to  the  coordinates 
of  an  1-dimensional  Euclidian  space.  He  specifies  the  best 
alphabets  of  two  and  three  dimensions  with  codebooks  from 
3  to  13  words.  Gilbert  is  plotting  the  performance  of  non¬ 
binary  systems  of  this  kind  in  an  “  efficiency  chart  ”  very 
similar  to  the  utility  chart  described  in  Part  I.  Indeed 
Gilbert’s  result  can  be  easily  plotted  in  a  utility  chart  of 
multilevel  systems  (Filipowsky  68d). 

Shannon  (48)  and  (later)  Gilbert  (52)  pointed  out  that  a 
good  alphabet  can  be  obtained  by  using  as  code  words  the 
vectors  to  all  the  lattice  points  which  are  separated  at 
least  unit  distance  from  one  another  and  which  are  contained 
inside  a  hypersphere  about  the  origin  of  the  1-dimensional 
space.  This  sphere-packing  problem  elicited  much  interest 
in  multidimensional  geometry  (Schaute  05  ;  Coxeter  48,  62  ; 
Sommerville  58).  Kharkevitch  (56)  and Poritsky  (57)  applied 
it  in  coding  theory.  Basore  (59)  and  Stutt  (59)  developed 
on  this  basis  the  regular  polyhedron  codes,  also  called  regular 
simplex  codes.  Stutt  (60)  explains  a  regular  simplex  in  1 
dimensions  as  a  general  body  bounded  by  1  +  1  intersecting 
hyperplanes  ;  this  is  the  analog  of  a  triangle  in  a  two-dimen¬ 


sional  space  or  a  tetrahedron  in  a  three-dimensional  space. 
If  the  edges  are  all  of  equal  length,  the  simplex  is  regular. 

Stutt  (60)  specifies  the  codebook  in  1-dimensional  space 
as  an  M-member  column  matrix  of  code  word  vectors,  which 
may  also  be  expressed  as  a  matrix  product  of  an  M-by-1 
matrix  of  projections  and  an  1-membor  column  matrix  of 
unit  coordinate  vectors. 

Balakrishnan  (61)  gave  a  rigorous  mathematical  analysis 
of  the  sphere-packing  problem.  Ziv  (62,  63),  based  on 
Fano’s  (61)  studies  of  the  general  discrete  memoryless  channel, 
investigated  the  coding  and  decoding  schemes  for  the  time- 
discrete  nonbinary  transmission  system  with  codebooks  of 
the  random  selection  type,  originally  suggested  by  Rice  (50). 

A  recent  encoding  process,  called  “  linear-real  coding  ”, 
published  by  W.  H.  Pierce  (66)  also  uses  a  codebook  matrix 
with  real  numbers.  In  this  case,  the  input  information  has 
been  systematically  smeared  out  over  the  whole  transmission 
interval.  The  transmission  signals  do  not  represent  one-and- 
only-one  code  word  during  a  given  transmission  interval, 
but  actually  a  superposition  of  all  code  words.  In  our 
terminology  we  have  to  classify  tho  linear-real  coding  process 
as  a  polysignal  system  and  not  as  a  single-signal  system. 

Contrary  to  Pierce’s  extremely  general  approach  one 
can  find  in  another  recent  publication  by  Slopian  (65a)  a 
rather  simple  but  still  very  efficient  encoding  procedure. 
It  is  called  permutation  modulation  because  the  codebook 
is  defined  by  a  single  codebook  word,  placed  as  W,  ,  and  by 
all  possible  permutations  of  this  first  word,  placed  as  W2  ,  W3  , 

.  .  .  ,  W.u  .  Figure  18  shows  a  number  of  typical  examples  of 
permutation  codebooks.  On  top  of  the  figure  is  the  general 
form  of  a  word.  The  yt  elemonts  can  be  any  real  numbers, 
not  necessarily  integers.  Zero  and  negative  values  can  be 
included.  Each  number  can  bo  repeated  any  number  of 
times.  There  are  two  types  of  these  codes  : 

(i)  Type  I  is  formed  by  placing  the  first  code  word  into 
its  natural  order,  i.e.,  lowest  numbers  to  the  left  and 
progressing  to  the  right  with  increasing  numbers, 
equal  numbers  being  next  to  each  other.  All  the  other 
code  words  are  formed  by  all  possible  permutations 
of  the  first  code  word.  Code  numbers  1,  2,  3,  belong 
to  this  type. 

(ii)  Type  II  codes  start  in  the  same  way  with  the  exception 
that  the  first  word  consists  of  non-negative  real 
numbers  only.  The  other  codes  are  then  formed  by 
permuting  the  order  of  the  elements  and  by  making 
all  possible  assignments  of  signs  to  the  elements.  Code 
numbers  4  and  5  are  of  this  type. 
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Figure  18.- — Examples  of  codebooks  for  permutation  modulation. 


The  two  equations  for  M,  and  M/f  give  the  size  (total 
number  of  code  words)  for  each  codebook. 

It  is  interesting  to  note  that  many  well-known  codes  are 
special  cases  of  such  permutation  codes.  Code  number  3, 
for  example, 'is  a  simplex  code  with  one  redundant  dimension. 
Slepian  points  out  that  the  condition, 

M,  +  M.,  g,  =  0  ,  (48) 

I  makes  a  code  with  k  =  2  (two  different  characters)  a  simplex 
code. 

Code  number  4  is  a  biorthogonal  code,  since  the  cross- 
correlation  between  any  two  words  is  zero,  except  for  the 
pair  formed  by  a  code  word  and  its  negative  (W,  and  \V5 , 
for  example).  Such  pairs  have  n  cross-correlation  coefficient 
of  —1.  If  a  code  is  formed  from  the  same  first  word  by 
permutation  only  (Type  I),  a  codebook  of  the  first  four  words 
results.  This  is  an  orthogonal  code  and  leads  particularly 
jto  a  nonbinary  system  that  we  called  multiple  time  shift 
keying,  also  known  as  quantized  pulse  position  modulation. 
That  and  other  kinds  of  orthogonal  and  biorthogonal  codes 
will  bo  discussed  in  section  K. 

Code  number  5  is  the  well-known  binary  PCM  code  with 
three  digits.  It  results  here  as  a  subclass  ofTyPe  1 1  permuta¬ 
tion  codes.  Slepian  made  a  search  of  about  100  codes, 
some  of  very  large  n  (up  to  100),  and  investigated  their 
performance  in  ideal  low-pass  systems,  comparing  such 
systems  with  the  upper  bound  of  Shannon’s  channel  capacity. 

A  report  by  Floyd  and  Nuttall  (65)  contains  a  very  general 
treatment  of  nonbinary  systems  from  which  we  represented 
a  specialized  version  in  fig.  16.  This  specialized  version  is 
called  by  the  authors  the  q-M-ary  system,  indicating  that 
they  use  a  codebook  of  M  words  consisting  of  sequences  of 
q-ary  characters.  The  largest  part  of  the  Floyd  and  Nuttall 
report  deals  then  with  a  further  specialization,  making  q  2. 
This  means  they  investigate  nonbinary  systems  using  wave¬ 
forms  which  are  synthesized  from  binary  sequences  (code 
words).  We  would  like  to  stress  the  importance  of  their 
report  to  research  workers  investigating  more  general  code¬ 
hooks,  similar  to  those  discussed  before,  but  using  integers  as 
codebook  characters. 


The  special  selection  of  signals  for  the  waveform  library 
will  he  discussed  separately  in  Part  IV  because  it  seems  that 
two  different  schools  of  thought  are  simultaneously  developing 
rather  different  and  novel  ideas.  Here  in  Part  III  we 
intend  to  concentrate  on  the  codebook  approach  only,  i.e.,  on 
the  effort  to  express  messages  in  a  sequence  of  numbers  with¬ 
out  regard  to  the  ways  of  expressing  these  numbers  in  band- 
limited  waveforms. 

J3.  Receiver  Fundamentals  for  Multidimensional  Systems 

Turning  now  to  the  receiver  side  of  a  multidimensional 
nonbinary  system,  one  notices  the  large  amount  of  research 
that  has  been  devoted  to  the  specifications  of  an  opt  imum 
receiver  design  for  analog,  binary,  and  nonbinary  signals. 
We  already  explained  in  Part  I  that  an  efficient  receiver  for 
nonbinary  transmission  signals  will  have  at  the  input  a 
rather  complex  analog  section,  called  the  signal  processor 
or  signal  extractor,  it  is  in  this  analog  section  that  much 
of  the  research  results  of  optimum  reception  theory  for 
analog  signals  can  be  applied — with  reservations,  however. 
We  ment  ioned  a  selection  of  references  to  research  on  mathe¬ 
matical  extractor  models  in  Part  1,  subsection  D3.  Part 
of  the  devices  used  in  those  mathematical  modeLs  will  be 
located  in  the  block  called  demodulator  matrix  in  fig.  16. 
This  demodulator  matrix  will  assume  quite  different  forms 
for  the  different  nonbinary  systems  and  for  the  different 
waveform  libraries  in  use.  Some  of  its  realizations  have 
already  been  discussed  in  Part  II  in  connection  with  the 
various  multistate  systems.  Regardless  of  the  specific 
circuits  in  a  multidimensional  system,  the  output  of  the 
demodulator  subsystem  must  give  the  best  possible  estimate 
of  which  one  of  the  waveforms  has  been  received  in  any 
subinterval  T,-  out  of  the  1  intervals  of  a  transmission  word. 

To  perform  an  efficient  estimation  operation,  the  demodula¬ 
tor  subsystem  must  receive  the  following  information  as 
input : 

(i)  The  complote  waveform  library,  with  clean  undistorted 
noise-free  waveforms. 
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(ii)  The  noisy  mutilated  waveforms,  ns  received  over  the 
channel,  and  after  having  been  processed  in  the  signal 
processor. 

(iii)  Timing  information  indicating  with  highest  precision 
the  beginning  and  the  end  of  each  subinterval  and 
any  additional  phasing  information  of  carriers  and 
subcarriers. 

(iv)  Any  additional  statistical  information  (long  term  and 
short  term)  about  the  status  of  the  channel  (noise, 
fading,  etc.),  hopefully  tagged  with  confidence  levels. 

(v)  In  advanced  systems,  any  additional  service  informa¬ 
tion,  such  as  code  numbers,  indications  of  message 
repetitions,  etc. 

Some  of  these  inputs  will  be  permanent  or  semipermanent 
and  thus  may  be  stored  in  memories  or  in  the  form  of  circuits 
(matched  filters)  within  the  demodulator  subsystem.  The 
waveform  library  falls  into  this  group.  Details  will  be 
discussed  in  Part  IV.  Other  inputs  will  be  transmitted  over 
the  channel  as  special  service  messages  ;  they  will  become 
available  to  the  demodulator  subsystem  via  the  decoder 
(servico  information).  Still  other  inputs  will  be  available 
from  special  circuits  in  the  receiver,  but  the  operation  of  these 
circuits  must  be  maintained  by  continuously  or  stepwise 
updating  their  status  (synchronization  via  phase  lock  loops). 

Finally,  it  will  be  necessary  to  apply  special  sensors  and 
computing  circuits  to  extract  the  statistical  information 
about  the  channel  either  directly  from  the  noisy  channel, 
or  indirectly  via  demodulator  and  decoder  from  the  noisy 
waveforms  and  from  corrected  message  errors. 

The  basic  nonbinary  systems  discussed  in  Part  II  do  not 
yet  provide  all  this  information  in  satisfactory  form.  Yet 
the  mathematical  models  used  in  the  analysis  of  these  basic 
systems  are  derived  under  the  assumption  that  all  this 
information  will  be  provided.  This  discrepancy  between 
theory  and  practice  presents  several  problems  to  the  practical 
engineer.  To  bridge  this  gap,  further  research  about  the 
receiver  subsystems  is  needed. 

No  matter  what  degree  of  sophistication  the  demodulator 
system  assumes  and  how  close  it  is  able  to  approach  an  ideal 
operation,  the  output  of  the  demodulator  system  will  always 
be  a  complete  or  a  simplified  cross-correlation  matrix  as 
shown  in  fig.  16.  If  the  receiver  can  bo  linked  to  the  trans¬ 
mitter  via  an  ideal  attenuator  (noise-free  case),  the  cross¬ 
correlation  matrix  should  be  an  ideal  matrix  containing 
“  sure  ”  coefficients.  This  ideal  matrix,  in  the  case  of  other 
codes  than  simplex  codes,  will  not  be  as  regular  as  the  one 
for  simplex  codes  (Floyd  and  Nuttall  65). 

The  noise-free,  distortion-free  case  is  not  a  realistic  model 
for  practical  systems.  It  is  evident  that  the  designer  must 
start  off  with  some  reasonable  assumption  about  the  noise 
level  at  the  output  of  the  demodulator  subsystem.  If  the 
characteristics  of  the  waveforms  (ratio  b d / q )  and  the  signal 
contrast  ( a)  at  the  receiver  input,  together  with  the  noise 
statistics,  permit  the  prediction  of  a  high  signal  contrast 
at  the  demodulator  output,  one  will  be  inclined  to  permit  a 
positive  high  value  of  the  maximum  cross-correlation  co¬ 
efficients,  counting  on  small  variations  of  the  output  pattern 
due  to  the  noise.  In  other  words,  if  the  external  noise  is  small, 
one  can  permit  a  certain  amount  of  internal  cross  talk  noise 
between  the  waveforms.  Conversely,  if  the  signal  contrast 
which  the  designer  expects  at  the  output  of  the  demodulator 
system  is  small  (i.e.,  if  the  noise  is  strong),  one  will  attempt 
to  make  the  maximum  cross-correlation  coefficients  in  the 
matrix  ns  small  as  possible  (e.g.,  zero  or  slightly  negative). 
This  will  leave  a  wider  margin  for  the  noise  before  final 
errors  in  the  decision  system  can  result. 

Turning  now  to  the  research  on  the  decider  and  decoder 
subsystem,  one  will  clearly  recognize  that,  the  simplest 
transfer  mode  of  extractor  results  after  a  maximum  like¬ 
lihood  decision  needed  an  accurate  analysis.  In  this  case  a 


multivariate  decision  is  made  at  the  end  of  each  interval 
T d  .  Nuttall  (62)  derived  an  expression  for  the  error  prob¬ 
ability  of  such  a  decision,  based  on  the  maximum  likelihood 
decision  rule.  He  assumes  that  the  demodulator  matrix  in 
fig.  16  performs,  during  each  waveform  interval  To  ,  a  cross¬ 
correlation  of  the  received  noisy  signal  with  replicas  of  all 
possible  waveforms.  Assuming  that  the  first  signal  has  been 
transmitted,  the  demodulator  will  receive  r(t)  =  s,  (t)  + 
n(t),  with  n(t)  represent  ing  the  noise  in  To  .  The  demod¬ 
ulator  matrix  will  have  q  outputs  of  the  form  : 


(t)dt 


Output  one  : 

T  T 

f  » 

x,  =  s,  (t)  st  (t)dt  +  s,  (t)  .  n 

Jo  Jo 

Other  outputs 

rTn  oTd 

x<  =  I  (t)  s,  (t)clt  +  |  s,  (t)  .  n(t)dt 


Assuming  that  all  waveforms  are  of  equal  energy,  Ed  ,  and 
equally  correlated,  one  can  substitute : 


x,  =  Eo  +  ya 
x<  =  Ed  A  +  y, 


y. 


s,  (t)  .  n(t)dt 


s(  (t)  .  n(t)dt 


(50) 


The  decision  rule  implies  that  a  correct  decision  will  be 
made  as  long  as  x,  (t)  is  larger  than  any  other  Xi  (t).  The 
probability  for  a  correct  decision  is  therefore  : 

Pc  =  Pr(Eo  (1  —  A)  +  y,  >  y2 ,  y, ,  .  . . ,  y„)  (51) 

I’c  can  be  calculated  by  integration  over  the  joint  probability 
density  function. 


<ly2  .  .  .  dy,  .  p(y,  ,  v2  ,  .  .  .  ,  y„)  (62) 


Under  certain  assumptions  it  is  possible  now  to  calculate 
the  probability  of  a  correct  decision  under  the  MLD  rule 
(Nuttall  62).  The  result  is  equation  53  : 


l  r+0°  , 

f 

/Ed(1  -  A)\ 

P,  = -  e-=-/2  .  P’-1 

z  •f  i 

V2 

/  2N„  / 

with  (53) 

P(x)=—  I"1  o-,2/Jdt 

V2 

The  function  P(x)  is  tabulated  in  many  mathematical  tables 
as  the  cumulative  distribution  function  of  the  normal  distribu¬ 
tion.  The  integral  of  equation  53  is  tabulated  in  the  form  : 

f  +  ao 

<!>(■/.)  P,_l  (z  -}-  a)  dz  (5-1 ) 

J  _oo 

in  the  following  references  (Birdsall  and  Peterson  54,  Urbano 
55,  D.  M.  Green  and  Birdsall  58,  Elliott  59). 

The  average  error  probability  for  one  character  to  be  in 
error  is  : 

ec  =  1  -  Pc  (55) 

Another  approach  to  the  same  problem  has  been  published 
by  Arthurs  and  Dym  (62)  in  the  same  year  as  Nuttall’s  paper. 
These  authors  arrive  at  an  upper  and  lower  bound  for  the 
character  error  ratio,  ec .  They  uso  t  he  distance  between 
message  points  as  the  key  variable,  while  Nuttall  uses  the 
cross-correlation  coefficient  A  as  the  key  variable  which 
expresses  the  characteristics  of  the  waveform  library.  A 
direct  comparison  between  the  results  of  the  two  indepen- 
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dently  developed  theories  requires,  therefore,  the  knowledge 
of  the  actually  used  waveforms  s,  (t)  .  .  .  s9  (t).  More 
research  will  be  necessary  to  perform  such  comparisons. 
When  they  become  available  it  will  be  useful  to  plot  the 
results  in  a  utility  chart  and  to  compare  them  for  any  given 
dimensions.  1  =  2BT»  ,  with  the  ideal  time-limited  system 

of  Shannon /Slepian  which  has  the  same  number  of  dimen¬ 
sions.  (Thje  ut  ility  curves  for  this  ideal  time-limited  system 
are  in  fig.  5  of  Part  I.)  Such  comparison  would  disclose 
the  efficiency  of  a  nonbinary  system  with  the  particular 
waveform  library. 

There  are  less  definitive  results  in  the  literature  regarding 
an  accurate  analysis  of  the  other,  more  efficient,  modes  of 
transfer  of  jthe  demodulator  results  to  the  decoder. 

The  decoder  research  followed,  for  a  long  period,  the  develop¬ 
ment  of  binary  decoders.  Many  interesting  nonbinary 
error-correcting  codes  are  known  but  decoding  algorithms 
have  not  been  devised,  except  in  very  simple  cases.  Floyd 
and  Nuttal)  ((>5)  present  a  general  approach  to  the  decoding 
problem  in  their  treatment  of  the  q-M-ary  nonbinary  system. 
This  follows  closely  the  lines  of  fig.  Ill;  full  use  is  made  of 
Nuttall’s  analysis  (G2)  of  the  MLD  demodulator  which  we 
discussed  above.  Their  final  decoder  is  again  a  maximum 
likelihood  decoder.  Floyd  and  Nuttall  developed  an 
upper  bound  for  the  final  error  probability  at  the  output 
of  the  decoder. 

An  interesting  recent  development  in  the  decoding  tech¬ 
nique  is  the  procedure  for  decoding  Slopian’s  permutation 
codes.  Wei  explained  these  codes  in  connection  with  fig.  18. 
The  particular  regular  structure  of  a  codebook  derived  by 
permutations  of  the  first  code  word  leads  to  a  very  simple 
decoding  algorithm  (Slepian  65a). 

The  decoder  for  Typo  I  codes  converts  a  received  sequence 
P\  ,  /S'2  f}'i  (fig.  16)  into  a  word  of  the  codebook  by 

replacing  the  i-th  largest  /3'  by  the  i-th  largest  number  in  the 
sequence  forming  the  first  word  of  the  codebook.  The 
decoder  for  Type  II  codos  converts  a  received  sequence  /i',  , 
/3'r  into  a  word  of  the  codebook  by  replacing  the  /?' 
that  is  i-th  largest  in  absolute  value  by  the  i-th  largest 
number  in  the  sequence  forming  the  first  word  of  the  codebook. 
The  j-th  term  in  the  sequence  thus  formed  is  then  given  the 
algebraic  sign  of  /}'j  (j  =  1,  2,  .  .  .  ,  1). 

In  fig.  19  we  show  an  example  fo  this  decoding  operation, 


comparing  it  with  a  maximum  likelihood  decoding  operation 
in  the  simplest  mode  (decision  after  demodulator)  and 
in  a  more  complex  mode  (largest  real  value  taken  from  the 
demodulator  for  each  /?'/)•  Figure  19  shows  at  the  top  the 
transmitted  code  word,  which  is  W8  taken  from  code  num¬ 
ber  2  in  fig.  18.  It  is  assumed  that  the  code  word  arrives 
in  heavily  mutilated  form  as  Wx  at  the  output  of  the  demod¬ 
ulator  matrix.  If  a  decision  is  made  after  the  demodulator, 
the  code  word  W „  would  result,  provided  the  thresholds 
are  at  0  and  1 .5,  halfway  between  the  three  acceptable  digits 

—  1,  -j- 1 ,  and  -(-2.  If  this  were  the  end  of  the  receiving  oper¬ 
ation,  one  can  see  that  only  two  digits  out  of  four  would 
be  correct,  though  an  error-detecting  device  could  be 
actuated,  indicating  that  the  resulting  word  is  not  a  word  of 
the  codebook. 

Figure  19  further  shows,  on  the  left  side,  the  12  words 
of  the  selected  codebook.  This  is  a  rather  arbitrarily 
selected  codebook  and  the  reader  can  verify  that  its  noise- 
free  cross-correlation  matrix  shows  cross-correlation  coeffi¬ 
cients  from  A4.  9  =  —1  to  A5,  8  =  +6/7.  Such  largo 
differences  may  be  desirable  in  special  cases,  especially 
when  the  input  message  probabilities  also  show  very  large 
differences.  The  second  column  shows  the  decoding  matrix 
when  decoding  the  word  W ,,  in  an  MLD  operation.  The 
products  of  corresponding  digits  are  listed.  For  example, 
the  cross-correlation  of  word  \V„  with  the  word  W,  is  per¬ 
formed  by  multiplying  the  first  digit  of  word  W „  (  —  1)  with 
the  first  digit  of  W,  (—1).  The  result,  +1,  is  noted  down. 
The  second  product  is  —1  x  —  1  =  +1  ;  the  third  product, 

—  1  x  +1  =  —  1;  the  fourth  product,  +1  X  +2  =  +2. 
The  sum  of  the  four  products  (+3)  must  be  divided  by  the 
energy  of  a  clean  signal  This  is  the  sum  of  the  squares  of 
all  four  digits  of  any  codehook  word,  in  this  case,  E  =  7. 
The  cross-correlation  coefficient  A„,  is,  therefore,  3/7  =  0.429. 

The  cross-correlation  of  W v  with  the  whole  codebook 
shows  that  there  are  three  coefficients  of  equal  and  highest 
positive  value.  The  third  matrix  in  fig.  19  shows  the  same 
procedure  when  applied  to  the  real  value  (undecided) 
received  word,  W*  .  In  this  case  the  correct  word  can  be 
clearly  identified  because  Axs  has  the  highest  positive  value 
of  all  cross-correlation  coefficients  (3.0/7  =  0.514).  At 

the  right  side  of  fig.  19  Slepian’s  decoding  procedure  is  applied  ; 
it  too  leads  to  the  correct  result.  However,  this  simple 
algorithm  can  bo  much  more  easily  instrumented  than  the 
MLD  decoding  operation,  particularly  when  the  codobook 
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Figure  19. — Comparison  of  decoding  procedures  for  permutation 
modulation. 
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is  very  large.  In  his  I9G5  paper  Slepian  (65a)  develops 
explicit  expressions  for  the  ei-ror  probabilities  of  Type  I 
and  Type  II  codes  and  submits  proof  that  the  decoding 
procedure  described  above  and  demonstrated  in  fig.  19 
minimizes  the  average  error  probability  per  transmitted 
word  under  conditions  specified  in  the  paper. 

Concluding  section  J  we  should  refer  to  an  applications- 
oriented  report  (Brumbaugh  60)  by  staff  members  of  Radio 
Corporation  of  America.  Their  study  on  coding  and  modula¬ 
tion  techniques  for  future  Astro-Communications  systems 
compares  open-loop  redundant  codes,  open-loop  book  codes, 
and  closed-loop  coding  techniques.  It  gives  information 
about  single  level  and  multilevel  phase  shift  keying  and 
frequency  shift  keying  methods  of  modulation.  The  study 
concludes  that  the  most  promising  coding  techniques  are 
book  codes  with  orthogonal  signals  combined  with  multilevel 
FSK  and  differentially  coherent  PSK  waveforms.  The  more 
advanced  nonbinary  systems  had  not  yet  been  considered 
in  this  study  of  August  1960. 

K.  Mathematical  Models  of  Multidimensional  Systems 

The  present  section  K  shows  how  mathematical  models 
of  these  coded  multidimensional  systems  can  predict  their 
efficiency.  Although  not  many  such  systems  are  in  an 
operational  status,  a  few  will  ultimately  emerge  as  the  highly 
efficient  systems  of  the  future.  The  scarce  experimental 
results  which  are  now  available  will  be  discussed  in  section  L. 


Kl.  The  Evolution  of  Mathematical  Models  for  Coded  Systems 
Prior  to  1957 

The  previously  mentioned  dual  approach  to  develop 
mathematical  modols  of  multidimensional  systems,  either 
along  the  coding  idea  or  along  the  waveform  library  idea, 
is  very  much  a  historical  fact. 

The  coding  approach  has  its  origin  in  the  error-detecting 
codes  for  teleprinters  that  emerged  in  the  second  quarter 
of  this  century.  In  particular,  the  evolution  of  the  seven- 
unit  code  (Sparks  and  Kreer  47),  with  its  constant  4  :  3  ratio 
between  spaces  and  marks  in  the  teleprinter  characters, 
paved  the  way  towards  a  systematic  investigation  of  binary 
codebooks.  Hamming  (50)  of  Boll  Telephone  Laboratories 
worked  for  many  years  on  an  extension  of  the  error-detecting 
parity  check  codes — previously  used  in  the  early  computing 
machines  (Alt  48) — -to  error-correcting  codes.  Shannon, 
in  his  famous  1948  publication,  credited  R.  W.  Hamming 
with  what  thus  far  is  the  first  error-correcting  code,  a  block 
code  of  seven  binary  digits,  including  three  check  digits. 
Golay  (49a)  published  a  short  note  on  digital  coding,  showing 
that  Hamming’s  seven-bit  code  can  be  extended  to  blocks 
of  2"— 1  length,  thus  providing  codes  of  15,  31,  63,  etc. 
binary  elements.  Still  more  important  for  the  purpose  of 
this  paper  is  Golay’s  extension  of  Hamming’s  block  codes 
to  blocks  of  nonbinary  elements.  In  the  same  year  also 
appeared  a  mathematical  paper  by  Foth  (49)  that  could 
very  well  be  the  oldest  contribution  to  the  famous  sphere¬ 
packing  problem  which  in  turn  leads  to  the  simplex  codes 
(section  J,  especially  Balakrishman  (61)).  Finally,  in  1950 
Hamming’s  famous  paper  on  “  error-detecting  and  error- 
correcting  ”  codes  defined  many  magnitudes  which  are 
characteristic  for  all  block  codes  and  which  are  essential 
for  the  application  of  binary  sequences  in  nonbinary  systems. 

Whereas  Hamming’s  paper  concentrated  on  bit-by-bit  deci¬ 
sion  and  on  logical  operations  for  correcting  errors,  R.  M. 
Fano  in  a  report  in  1951  and  in  a  paper  in  1952  prepared  the 
way  for  the  correlation  detection  of  complete  code  words  in 
place  of  the  bit-by-bit  decision  followed  by  logical  decoding. 
This  step  clearly  converts  binary  data  transmission  systems 
with  block  codes  to  nonbinary  data  transmission  systems 
by  considering  each  transmitted  code  word  as  one  trans¬ 
mission  signal  out  of  a  collection  of  more  than  two  trans¬ 
mission  signals.  The  fact  that  the  transmission  signal 


consists  of  binary  elements  Is  merely  a  design  feature  and 
not  a  fundamental  characteristic  of  the  system. 

These  first  results  of  the  coding  researchers  led  to  the 
hope  that  the  way  to  reach  Shannon’s  channel  capacity  C 
(section  C2)  could  ho  via  such  sequences  of  binary  elements. 
We  already  referred  in  sections  A  and  D1  to  the  research 
results  of  Shannon  (48)  and  Rice  (50),  proving  that  the  ideal 
channel  capacity  C  (equation  17)  can  be  reached  in  the  limit 
when  using,  as  transmission  waveforms,  sequences  of  in¬ 
creasing  length  consisting  of  real  numbers  drawn  from  a 
normal  universe.  The  problem  facing  the  mathematicians 
in  1952  was  to  devise  an  algorithm  for  artificially  creating 
such  random  signals,  i.e.,  to  generate  pseudorandom  signals. 
In  1954  Golay  announced  his  results  on  binary  coding, 
indicating  that  symbol-correcting  codes  such  as  the  Hamming 
codes  are  less  likely  to  reach  C  than  message-correcting  codes. 
The  former  class  of  codes  uses  special  check  digits  in  addition 
to  information  digits  ;  i.e.,  they  are  defined  by  sets  of  logical 
equations  for  calculating  the  check  digits.  The  message- 
correcting  codes,  on  the  contrary,  are  defined  by  a  code¬ 
book  ;  there  is  no  need  to  calculate  any  special  redundant 
digits.  Also  in  1954  Elias  showed  that  the  symbol-correcting 
codes  (also  called  systematic  codes  or  digit  codes)  can  be 
extended  by  an  iterat  ion  of  simple  error-correcting  and  detec¬ 
ting  codes.  This  process  is  easy  to  instrument  and,  by 
increasing  the  number  of  check  digits  and  the  complexity 
of  the  process,  the  designer  can  reach — for  any  given  noisy 
channel — an  error  probability  as  low  as  ho  may  desire. 
Elias  called  the  process  “  error-free  coding  ”.  Yet  a  system 
using  these  codes  cannot  reach  this  error- free  operation 
at  channel  capacity  C.  Feinstein  (54,  55)  and  Shannon  (56) 
proved  that  this  zero-error  capacity  is  not  only  a  characteristic 
of  Elias’s  check  digit  codes  but  a  very  fundamental  magnitude 
that  can  be  computed  for  many  channel  models.  Reed  (54) 
showed  that  a  special  class  of  message-correcting  codes  has 
multiple  error-correcting  capabilities.  Those  codes  are  now 
well  known  under  the  designation  of  Reed-Mnllor  (RM)  codes, 
Reed  having  based  his  algorithm  on  the  somewhat  earlier 
work  of  Muller  (53).  Reed  should  also  receive  credit  for 
introducing  a  majority  testing  decoding  procedure,  now  better 
known  as  maximum  likelihood  decoding  (section  M)  and 
closely  related  to  the  multiple-alternative  detection  method  of 
Middleton  and  Van  Meter  (55). 

Studies  during  these  first  ten  years  (1947-1957)  of  the  evolu¬ 
tion  of  the  coding  theory  concluded  that  : 

(i)  Encoded  sequences  of  numbers  are  useful  as  trans¬ 
mission  messages. 

(ii)  The  decoding  process  should  be  a  maximum  likelihood 
decoding  (MLD)  process  rather  than  a  bit-by-bit 
decision  process  with  subsequent  logical  error  correc¬ 
tion. 

(iii)  The  length  of  the  code  words  (number  of  columns 
in  the  codebook  matrix)  should  be  as  long  as  econ¬ 
omically  feasible. 

(iv)  The  size  of  the  code  word  alphabet  (number  of  rows 
in  the  codebook  matrix)  should  be  matched  to  the 
SNR  of  the  channel. 

(v)  When  the  right  code  and  the  optimum  dimensions 
of  the  codebook  matrix  (ratio  of  the  number  of  columns 
to  rows)  are  selected,  it  is  possible  to  arrive  at  error- 
free  operation. 

The  unsolved  problems,  however,  were  numerous  : 

(i)  Are  binary  sequences  or  nonbinary  sequences  prefer¬ 
able  ? 

(ii)  What  algorithms  are  available  to  design  codebooks  ? 

(iii)  How  close  can  the  transmission  rate  approach  the 
channel  capacity  for  any  class  of  systems  (special 
selection  of  codes,  basis  of  the  number  system, 
selection  of  the  decoding  and  decision  procedure)  ? 
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(iv)  How  far  will  the  performance  of  any  given  system 
be  below  the  ideal  system  when  a  maximum  error 
ratio  is  specified  ? 

(v)  What  is  the  cost  (circuit  complexity)  and  the  reliability 
of  any  design  coming  close  to  the  optimum  perfor¬ 
mance  ? 

When  summarizing  the  efforts  prior  to  1957  to  find  suitable 
waveform  libraries  for  multidimensional  nonbinary  data 
transmission  systems  (section  M),  one  must  conclude  that 
no  single  approach  was  available  that  could  directly  lead 
to  an  engineering  model.  Yet  there  were  many  serious 
efforts  to  combine  the  coding  approach  with  the  waveform 
library  approach  to  create  the  concept  of  a  truly  efficient 
system.  These  overall  efforts  are  represented  by  two  books 
and  by  twd  review  reports. 

The  book  by  R.  M.  Fano  (61)  reflects  in  consolidated  form 
the  progress  of  both  approaches  during  the  1950’s.  During 
this  decade  Professor  Fano  made  significant  contributions 
toward  multidimensional  data  systems,  both  as  professor 
and  as  research  worker  at  Massachusetts  Institute  of  Tech¬ 
nology  in  Boston.  The  second  book  is  the  theory  of  optimum 
noise  immunity  by  the  Russian  engineer  and  scientist, 
V.  A.  Kotel’nikov  (59a). 

Tho  two  review  reports  are  the  report  by  Nut  tall  (62)  with 
72  reforendes  and  a  more  recent  report  by  Gibson  (66), 
with  many  charts  suitable  for  practical  tradeoff  calculations. 

All  the  efforts  discussed  above  laid  the  ground  work 
for  a  number  of  more  definitive  mathematical  models  of 
systems  based  on  largo  codebooks. 

K2.  Mathematical  Models  Since  1957 

Tho  most  important  of  tho  five  unsolved  probloms  that 
we  listed  as  being  attacked  around  1957  was  tho  search 
for  algorithms  to  encode  the  input  information  into  suitable 
codebooks.  This  search  concentrated  on  binary  sequences, 
since  their  behaviour  was  best  known  and  since  the  binary 
logical  circuit  technique  was  well  developed.  In  terms  of 
fig.  16  this  means  that  only  tho  letters  A  and  B  would  bo 
used  ns  elements  of  the  codcbook  and  of  the  decoder  matrix. 
The  waveform  library  would  have  only  two  members  (A 
and  B)  ;  tho  cross-correlation  matrix  in  the  receiver  would 
havo  only  two  columns,  marked  A  and  B.  We  recall  that 
one  of  the  problems  that  had  been  solved  prior  to  1955  was 
the  question  of  the  optimum  decoding  procedure.  Theory 
had  proved  that  MLD  or  decoding  by  correlation  was  the 
optimum  decoding  mode. 

To  implement  a  data  transmission  system  that  uses 
binary  code  words  ns  signals,  the  designer  makes  a  number 
of  selection:;.  Most  important  is  the  selection  of  the  code 
length  (1),  i.e.,  the  number  of  binary  elements  (also  called 
chips)  in  each  code  word.  Also  important  is  the  selection 
of  tho  number  of  code  words  (M).  The  ratio  of  1/M  indicates 
the  redundancy  of  the  codo ;  it  is  a  measure  for  the  number 
of  “  forbidden  ”  words.  If  M  =  2',  the  codebook  is  com¬ 
plete  :  all  code  words  that  can  be  formed  from  1  chips  are 
in  tho  codebook.  Naturally,  such  a  system  would  bo  identical 
with  an  uneoded  binary  system.  If  M  =  1,  the  codebook 
matrix  is  a  square  matrix.  It  will  bo  shown  below  that 
very  useful  orthogonal  codes  are  known  in  this  form.  Most 
of  the  known  systematic  codes  (error-correcting  codes)  have 
1  <  M  <  2'.  They  can  be  used  in  the  manner  indicated 
above  and  will  operate  as  message-correcting  codes. 

The  key  problem  to  be  solved  before  any  design  can  start 
is  naturally  the  exploration  of  all  possible  codebooks  and  their 
encoding  algorithms.  Since  1955  many  research  results  have 
been  published.  The  most  general  results,  or  those  of  special 
tutorial  value,  will  be  mentioned  in  this  section. 

By  1955  il  was  well  known  that,  periodical  binary  sequences 
coidd  be  generated  with  the  help  of  shift  registers  and  a  binary 
feedback  logic.  Zierler  (55)  summarized  the  theory  of  such 
shift  register  generators  and  Golomb  (55)  investigated 


the  randomness  properties  of  the  sequences  that  could  be 
generated  with  the  devices.  Some  sections  of  this  classical 
report  deal  also  with  sequences  of  nonbinary  chips.  Huffman 
(56)  established  a  theory  for  the  encoding  of  a  finite  sequence 
of  binary  information  digits  with  the  help  of  a  linear  binary 
sequence  filter.  At  the  transmitter  the  information  sequence 
is  inserted  into  the  filter  and  is  immediately  followed  by 
another  sequence ;  for  example,  an  all-zero  sequence, 
which  is  known  to  tho  receiver  in  advance  of  the  transmission. 
Tho  output  of  tho  filter  becomes  the  soquenco  to  be  trans¬ 
mitted.  An  inverse  filter  is  utilized  at  the  receiver  and  the 
deviation  of  tho  additional  sequence  from  its  (a  priori ) 
standard  form  can  be  utilized  for  the  subsequent  correction 
of  transmission  errors.  Thus  Huffman  succeeded  in  relating 
the  codebook  to  tho  structure  of  a  digital  filter  and  to  tho 
use  of  an  additional  binary  sequence  known  to  both  stations. 
Huffman  also  discusses  an  extension  of  this  idea  to  other 
than  modulo-two  number  systems.  Sequences  with  non¬ 
binary  chips  may  therefore  uso  the  same  basic  procedure, 
provided  that  nonbinary  logical  elements  are  available. 
Birdsall  and  Ristenbatt  (58)  prepared  an  excellent  tutorial 
introduction  to  linear  shift-register-generated  sequences. 

In  1959  we  find  strongly  increased  interest  in  shift  register 
sequences  and  in  their  theory  (Zierler  59).  Certain  properties 
make  them  attractive  for  communication  systems  (Campbell 
59),  and  renewed  progress  in  tho  theory  of  linear  sequential 
networks  (Elspas  59)  prepares  the  way  to  close  the  gap 
between  the  theory  of  systematic  codes  (Abramson  59)  and 
the  theory  of  block  codes  and  group  codes  (Fontaine  and 
Peterson  59,  Slepian  60).  The  latter  class  is  primarily 
destined  to  serve  as  codebooks  in  multidimensional  systems. 
The  first  book  on  error-correcting  codes  was  published  by 
W.  W.  Peterson  (61).  Although  basically  concerned  with  the 
theory  and  the  procedures  for  bit-by-bit  binary  symbol 
decoding,  it  is  valuable  for  multidimensional  nonbinary 
systems  as  a  review  of  the  coding  theory  of  tho  1950’s. 
Independently  but  concurrently  published,  wo  note  the 
first  part  of  a  comprehensive  investigation  of  cyclic  codes 
by  Elspas  and  Short  (61),  performed  under  government 
contract.  The  final  report  was  finished  in  1962.  The 
first  comprehensive  review  of  the  results  of  the  codebook 
approach  as  a  means  to  create  a  large  alphabet  of  waveforms 
in  the  form  of  binary  sequences  was  also  published  as  a 
report  by  the  Jet  Propulsion  Laboratory  in  Pasadena, 
California  (Baumert  et  al  61).  An  expanded  version  of  this 
report  was  later  made  available  in  book  form  (Golomb  et  al 
61).  These  surveys  of  the  state-of-the-art  in  tho  theory 
of  error-correcting  codes  and  of  the  evolution  of  tho  binary 
codebook  approach  toward  multidimensional  communication 
systems  paved  tho  way  for  specialized  investigations  of  some 
crucial  problems.  Eisenstadt  (61)  concentrated  on  the 
autocorrelation  characteristics  of  pseudonoise  sequences ; 
Wolf  (63)  explored  the  effects  of  filtering  a  random  binary 
sequence  with  certain  finite  memory  filters  (both  linear  and 
non-linear)  ;  Roberts  (63)  consolidated  the  information 
about  linear  ami  nonlinear  shift-register  generators ;  and 
Gilbert  (63)  considered  the  application  of  binary  sequences 
as  carriers  in  asynchronous  multiplexing  systems,  arriving 
at  a  class  of  cyclically  permutable  error-correcting  codes 
for  this  purpose. 

Since  1963  many  excellent  papers  have  been  available; 
some  of  them  are  applications  oriented,  while  others  are 
still  exploring  tho  remaining  theoretical  problems.  A  few 
contributions  of  general  interest  are  mentioned  hero. 

McWilliams  (64)  treats  the  practical  codebook  design 
problem  of  locating  cyclic  codes  with  a  fixed  block  length 
and  a  fixed  number  of  parity  checks,  provided  such  codes 
exist.  A  handbook  of  communication  codes  is  published 
(64a),  the  synthesis  of  binary  sequences  with  arbitrary 
realizable  autocorrelation  functions  is  attacked  (Lovitt  and 
Wolf  65),  a  systematic  classification  of  sequence-generating 
methods  into  five  basic  classes  emerges  (Brothman  et 
al  65),  and  cyclic  error-locating  codes  are  compared  with 
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many  other  known  codebooks  for  their  merits  in  multi¬ 
dimensional  communications  systems  (Goethals  67). 

We  recently  experienced  the  beginning  of  an  investigation 
of  orthogonal  codes  with  nonbinary  elements  (S.  H.  Chang 
66a).  More  will  be  said  about  this  important  branch  of 
multidimensional  nonbinary  systems  in  subsection  K7. 

The  references  in  section  K  which  wore  discussed  in  histo¬ 
rical  order  give  a  fair  cross  section  of  the  vast  literature 
on  the  research  in  codebooks  and  coding  (or  decoding) 
algorithms.  This  research  resulted  in  specific  mathematical 
models,  discussed  in  the  following  subsections  : 

K3  Models  of  multidimensional  channels  with  fixed 
dimension 

K4  Models  of  channels  with  codebooks  generated  by 
cyclic  sequences 

K5  Models  of  channels  with  orthogonal  binary  codebooks 

K6  Models  of  channels  with  suborthogonal  binary  code¬ 
books 

K7  Models  of  channels  with  nonbinary  codebooks 

K8  Models  of  channels  with  simplex  codebooks. 

It  is  important  to  notice  that  these  various  classes  of 
models  are  not  mutually  exclusive  ;  rather,  t  hey  are  grouped 
for  reasons  of  convenience  or  historical  evolution.  We  are 
still  in  the  midst  of  the  research  period  for  codeboolcs  and 
it  is  too  early  to  recognize  the  most  logical  way  or  the  most 
natural  way  of  classifying  codebook  models.  For  an  ordering 
of  communications  channels  in  general,  the  reader  is  referred 
to  Shannon  (58)  and  to  T.  T.  Chang  and  Lawton  (62). 

K3.  Mathematical  Models  of  Multidimensional  Systems  with 
Fixed  Dimensions 

Building  on  Feinstein’s  (57)  theory  of  memoryless  channels 
and  on  Shannon’s  (59)  theory  of  optimal  codes  in  a  Gaussian 
channel,  Slepian  (63)  arrived  at  a  very  general  model  for 
channels  with  fixed  dimension.  We  discussed  Slepian’s 
model  in  section  D1  and  we  plotted  his  results  for  codes  of 
5,  25,  or  101  dimensions  as  utility  curves  in  fig.  5.  One 
year  earlier  than  Slepian’s  paper,  research  workers  of  New 
York  University  arrived  at  similar  results  using  specific 
binary  codes  (S.  S.  L.  Chang  et  al  62).  Many  of  the  results 
of  this  research  and  of  related  papers  have  recently  been 
consolidated  by  Slepian  (68)  in  his  theory  of  group  codes 
for  the  Gaussian  channel.  Many  results  of  this  1968  paper 
are,  as  Slepian  himself  states  on  page  578,  based  on  “  the 
author’s  Bell  Telephone  Laboratories  report  of  May  7,  1951,  a 
document  that  received  only  limited  circulation  outside 
the  laboratories  ”. 

Slepian  defines  the  Gaussian  channel  as  a  model  that  corres¬ 
ponds  to  the  codebook  of  1  dimensions  shown  in  fig.  16, 
whereby  each  code  word  (message)  corresponds  to  a  vector 
in  an  1-dimensional  Euclidean  space  (signal  space).  The 
transmitted  signal  is  a  bandlimited  version  of  this  message 
vector,  while  the  received  signal  consists  of  the  sum  of  the 
sent  vector  and  a  noise  vector.  The  components  of  the 
noise  vector  are  independent  Gaussian  variates  with  mean 
zero  and  a  variance  that  depends  on  X  „  ,  the  noise  power 
density  in  the  channel. 

Slepian  uses  equal  energy  block  codes  of  size  M  as  codebooks. 
This  means  that  the  codebook  has  1  elements  and  M  code 
words,  exactly  as  shown  in  fig.  16.  All  codewords  have  the 
same  length,  i,  and  they  represent  points  in  the  vector  space. 
The  equal  energy  condition  implies  that,  all  these  vectors 
have  their  termini  on  a  hypersphere  of  radius  VlS0  ,  with 
Sc  being  the  average  power  of  the  code.  Slepian’s  theory 
is  restricted  to  codebooks  with  M  >  1 ;  it  is  primarily  con¬ 
cerned  with  a  class  of  equal-energy  block  codes  called  group 
codes.  Although  the  precise  definition  of  a  group  code 
involves  mathematical  expressions  of  the  theory  of  group 
representations  (Boomer  63),  Slepian  describes  a  group 
code  as  a  codebook  with  important  symmetry  properties 
that  cause  all  the  code  words  in  that  group  to  be  on  equal 
footing.  Most  codes  that  have  been  investigated  for  the 
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Gaussian  channel  are  group  codes  ;  it  is  likely  that  any 
code  used  in  practice  will  be  of  this  type.  Group  codes 
in  general  require  a  q-ary  waveform  library,  but  the  special 
case  q  =  2  is  of  high  interest  .  All  binary  signalling  alphabets 
discussed  by  Slepian  (56)  are  group  codes  in  the  sense 
of  his  latest  theory. 

Although  Slepian’s  theory  of  1968  will  prove  to  be  of 
fundamental  importance,  the  author  himself  concludes : 
“  The  development  of  this  subject  is  clearly  incomplete  :  We 
have  raised  more  questions  than  we  have  answered  ”. 
In  this  paper  we  therefore  face  the  problem  of  explaining 
the  state-of-the-art  of  codebook  models  without  the  benefit 
of  an  all-inclusive  theory.  We  shall  do  so  by  converting 
to  our  terminology  some  of  the  equations  that  have  been 
derived  in  the  book  by  Wozencraft  and  Jacobs  (65),  hence¬ 
forth  to  be  abbreviated  WJ. 

The  discussions  of  channel  models  with  codebooks  of 
fixed  length  may  start  with  the  special  case  1  2,  since  the 

one-dimensional  models  have  been  discussed  in  Part  II 
of  this  paper.  For  the  two-dimensional  case,  C.  L.  Weber 
(65)  showed  that  the  regular  M-gon  is  globally  optimal. 
This  result  is  in  line  with  the  well-known  fact  (Shannon 
59)  that  the  optimal  choice  for  a  two-dimensional  codebook 
is  to  space  the  M  signal  vectors  equally  around  the  unit 
circle.  An  analysis  of  the  performance  of  systems  operating 
with  codebooks  of  this  kind  is  given  by  Cahn  (59). 

For  any  1  >  2,  the  choice  of  I  and  M  is  the  problem.  Is 
it  preferable  to  use  a  long  codebook  with  only  a  few  code 
words  (i.e.,  1  >  M),  a  square  codebook  (1  =  M),  or  a  codebook 
with  a  high  and  short  matrix  (1  <  M)  ? 

To  answer  these  questions  we  turn  first  to  page  290  of 
WJ’s  (65)  analysis  of  block-orthogonal  signalling.  There 
we  find  equation  5.12  giving  an  upper  bound  for  the  word 
error  probability  P[c]  of  a  mathematical  model  representing 
any  set  of  M  equally  likely  equal  energy  orthogonal  signals. 
Converted  to  the  notations  of  this  paper,  the  upper  bound 
for  a  general  orthogonal  model  is  as  follows  : 

°»<  (M-1)q{^[  (56) 

where  ow  is  the  probability  that  a  received  code  word  will 
be  in  error.  Let  us  now  assume  that  each  code  word  carries 
n  =  log-  M  bits  of  information,  since  this  information  is 
required  for  monosignal  systems.  The  word  error  may 
then  cause  any  number  of  output  bits  up  to  n  to  be  in  error. 
If  the  channel  is  completely  symmetrical,  a  word  error  may 
convert  the  correct  word  with  equal  probability  into  any 
one  of  the  other  M  —1  words.  In  this  case  the  word  error 
ratio  is  related  to  the  bit  error  ratio  on  the  right  side  of 
equation  2,  employing  the  equality  sign  : 

2eB  =  e„  (57) 

The  Q  function  is  the  same  as  previously  defined  (fig.  8b). 
For  the  numerical  evaluation  we  used  tho  tables  on  pages 
966-973  in  the  “  Handbook  of  Mathematical  Functions  ” 
(Abramowitz  and  Stegun  64),  which  are  available  for  large 
arguments  (up  to  500).  The  utility  u  was  defined  in  Part  I 
and  has  been  used  throughout  this  paper. 

Because  we  are  interested  in  largo  values  of  M,  we  may 
make  a  further  simplification  and  replace  M  —  1  by  M  in 
equation  56.  This  leads  to  : 


In  logarithmic  measures  we  may  now  express  this  inequality 
as  an  implicit  function  in  u.  We  continue  under  tho  assump¬ 
tion  M  =2"  (monosignal  systems). 

0.30103  (1  —  n)  +  log10  c/i  <  log,,,  Q  j^/~}  (58a) 
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To  get  u  as  a  function  of  D — i.e.,  to  derive  the  equation  of  a 
utility  curve  (for  fixed  e«  as  curve  parameter) — one  must 
assume  the  bandwidth  requirement.  An  idealistic  assump¬ 
tion  (leading  to  an  upper  bound  for  the  bit  density)  is  the 
maximum  signal  base  that  any  orthogonal  set  may  reach  : 

M  =  2BTS  (59) 

This  na  ans  that  a  time  interval  of  1/2B  will  be  available 
for  each  dimension  of  the  multidimensional  signals  or  for 
each  element  of  a  code  word  in  case  the  elements  will  not 
be  further  encoded  into  a  waveform  library.  This  ideal 
assumption  corresponds  to  a  transmission  rate  of  2W  numbers 
or  samples,  i.e.,  to  an  operation  at  the  Nyquist  rate.  Equa¬ 
tion  59  can  be  expressed  in  terms  of  D  (the  bit  density)  and 
of  n  (the  number  of  information  bits  carried  by  one  mono¬ 
signal)  : 

D"  =  ndb  -  3.0103  (n  —  1)  (59a) 

Curves  '-an  now  be  plotted  by  assuming  a  constant  value 
for  ec  and  a  set  of  values  for  n.  Equation  58a  yields  the  u 
value  (in  numbers)  and  equation  59a  yields  the  D  value 
(directly  in  decibels).  Such  curves  are  shown  in  fig.  20. 
Because  of  the  approximation  of  58  versus  56,  the  curves 
are  unreliable  for  small  n  values. 


theory  must  therefore  investigate  codebooks  with  fixed 
values  of  1,  the  number  of  dimensions. 

Again  it  is  Wozencraft  and  Jacobs  who  give  us  a  convenient 
model  for  multidimensional  channels  with  fixed  dimensions, 
hut  now  with  a  variable  number  of  code  words  per  dimension 
(M  5*  1).  Notice  that  M  ;•>  1  will  lead  to  the  so-eallcd 
suborthogonal  systems  (the  cross-correlation  coefficients  bet¬ 
ween  cotie  words  will  be  positive)  while  M  <g  1  will  lead  to 
the  hyperorthogonal  systems  (systems  with  extremely  wide 
bandspreading)  for  which  the  cross-correlation  coefficients 
between  code  words  will  be  negative. 

WJ’s  model  uses  the  Q  function  (i.e.,  the  probability 
integral  for  the  Gaussian  distribution)  and  is  mathematically 
rather  complex.  In  fig.  2.36  on  page  83  of  WJ  (65),  one  may 
find  two  approximations  to  the  Q  function,  one  of  them 
using  the  simple  Gaussian  function  y  =  exp  (— x2).  It 
is  this  approximation  that  can  easily  be  used  to  derive  a 
conservative  lower  bound  on  the  utility  for  all  classes  of 
codebooks  and  for  constant  error  ratio.  Again  we  take 
this  bound  from  equation  5.12  on  page  290  of  WJ  : 


The  codebook  with  M  >  1  (vertical  rectangle)  will  bo  more 
desirable  in  most  applications  than  the  codebook  with 
M  <  1  (horizontal  rectangle).  The  reason  is  that  the  bit 
density  can  not  go  below  something  like  —50  or  —60  db 
because  thfe  bandwidth  requirement  becomes  excessive  for 
even  very  moderate  information  transmission  rates.  If 
one  desires  to  increase  the  utility  without  decreasing  the 
bit  density,  one  can  do  so  only  by  increasing  both  the  length 
of  the  code  (1)  and  the  number  of  code  words.  In  mono¬ 
signal  transmission  the  number  of  code  words  increases 
exponentially  (to  the  base  2)  with  the  length  of  the  code 
word,  to  keep  the  ratio  of  1/n  (and  therefore  the  bit  density) 
constant.  Thus  it  is  clear  that  the  codebooks  will  result 
in  higher  and  higher  rectangles  (M  g>  1),  when  trying  to 
improve  the  utility  without  changing  the  bit  density.  The 


Wozencraft  and  Jacobs  show  how  this  bound  can  be  applied 
to  the  special  case  of  binary  elements  (q  =  2  in  lig.  16). 
Assuming  further  that  the  two  waveforms  of  this  degenerate 
waveform  library  will  be  antipodal  to  each  other,  WJ  arrive, 
in  equation  5.38c  on  page  203,  at  the  following  mathematical 
model  for  systems  with  codebooks  consisting  of  binary  elements, 
written  in  WJ  notation  : 

pTri  <  2“a'K~".v]  (61) 

In  this  notation,  P[<]  corresponds  to  the  word  error  ratio 
(e,„)  of  this  paper.  N  is  the  number  of  dimensions  corres¬ 
ponding  to  1 ;  H.v  is  the  transmitter  rate  in  bits  per  dimension, 
corresponding  to  n/1 ;  and  R„  is  called  the  exponential 
bound  parameter.  R  „  establishes  a  limit  for  improvement. 
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As  long  as  Tt,v  <  R»  ,  the  word  error  ratio,  e,„ ,  can  be  made 
arbitrarily  small  by  taking  1  sufficiently  large.  R„  can  be 
expressed  in  terms  of  the  magnitudes  used  in  this  paper 
by  using  WJ  equation  5.36b  on  page  303,  and  when  intro¬ 
ducing  pc  as  the  signal  base  required  for  one  dimension. 
Pc  =  0.5  is  the  minimum  value  for  pc  (Nyquist  rate)  : 

R„  =  1  —  log2  |^1  +  exp  ^  J  (02) 

In  this  equation,  D  is  the  bit  density  (equation  1 1 )  and 
u  is  the  utility  (equations  13  to  15).  Combining  equations  61 
and  62,  inserting  the  magnitudes  used  in  this  paper,  and 
replacing  the  unequal  sign  by  an  equal  sign  lead  to  the  fol¬ 
lowing  equation  for  a  very  conservative  upper  bound  of 
the  utility  of  an  ideal  monosignal  system  with  a  codebook 
of  M  =  2"  code  words  and  a  code  longth  of  1  =  n//?c  D  : 


u,a  =  ptt,  +  D,ii.  _  io  logl0  [-  In  (2*-  -  1)]  (63a) 

The  constant  L  is  defined  by  the  equation  : 

L  =  —  (log,  Eb  +  1)  -  pc  D  +  1  (64) 

n 

Figure  21  shows  several  curves  following  equation  63  with 
the  parameters  ec  =  ew/2  =  10  3  (solid  lines)  and  on  =  10_“ 
(dot-dash  lines),  and  with  parameters  n  =  3,  5,  10,  20,  30,  50. 
Curves  of  constant  1  values  are  shown  as  dashed  lines  for 
cn  =  10-3only.  They  do  not  apply  for  the  lower  error  ratio 
of  10"  *.  All  the  curves  of  fig.  21  are  plotted  for  the  minimum 
character  signal  base  pc  =  0.5  (Nyquist  rate). 

For  interpreting  the  meaning  of  fig.  21,  assume  that  some 
one  designed  an  ideal  conceptual  system  with  a  binary 
codebook  of  1  =  40,  M  =  210  =  1024,  operating  at  the 
Nyquist  rate  of  two  dimensions  per  Hertz  bandwidth  and 
using  perfect  maximum  likelihood  detection  in  a  Gaussian 
channel.  The  designer  can  then  anticipate  operating  with 
an  error  probability  of  one  bit  in  one  thousand  at  the  output 
of  his  system.  This  is  characterized  by  the  operating  point 
P  in  fig.  21. 


There  are  several  important  aspects  that  should  be  remem¬ 
bered  when  using  a  utility  chart  of  an  ideal  mathematical 
model  such  as  fig.  21,  and  when  selecting  an  operating  point 
such  as  P  : 

(i)  The  mathematical  model  always  makes  idealized 
assumptions  that  cannot  be  completely  realized  in 
engineering  implementations.  In  all  the  models 
mentioned  in  this  section  we  assume  distortion-free 
circuits,  absolutely  perfect  synchronization,  perfectly 
generated  signals  that  correspond  exactly  to  their 
mathematical  definition,  etc. 

(ii)  The  mathematical  model  may  have  been  defined 
purposely  for  a  worst  limiting  case.  The  models 
of  figs.  20,  21,  and  22  use  all  the  worst  possibility 
when  converting  word  errors  into  bit  errors.  We 
assume  that  on  the  average,  half  of  the  bits  of  an 
output  word  will  be  in  error  if  the  word  itself  is  in  error. 

(iii)  The  mathematical  model  will  usually  be  valid  only 
for  certain  statistical  characteristics  of  the  input 
parameters.  In  all  our  models  we  assume  that  all 
possiblo  input  messages  will  occur  with  equal  proba¬ 
bility. 

(iv)  The  mathematical  model  may  be  representative  for  a 
class  of  systems  and  not  for  a  single  special  system. 
This  is  the  case  for  the  models  represented  by  figs. 
21  and  22.  They  actually  represent  an  average  of  all 
possiblo  binary  codebooks,  as  explained  on  pages  298- 
307  of  WJ*  (65).  Some  codebook  configurations  may 
therefore  yield  better  results  ;  others  will  yield  worse 
results.  More  restrictive  models  will  be  needed  to 
describe  the  ideal  performance  of  any  special  codebook 
configuration. 

Having  all  the  above  limitations  in  mind,  one  can  still 
derive  great  benefits  from  the  model  represented  by  fig.  21. 
It  is  also  interesting  to  note  (hat,  when  keeping  the  code 
length  constant  (for  example,  1  =  40)  and  when  increasing 
the  number  of  code  words  considerably  (for  example  to  224), 
one  can  find  another  operating  point  for  the  same  utility 
(—5.3  db)  but  at  much  higher  bit  density  (D  =  +0.7  db). 

On  the  other  hand,  when  the  bandwidth  requirement  for  a 
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Figure  21.— Ut  ility  curves  of  systems  with  binary  codebooks  of  various 
dimensions. 
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Figure  22.— Comparison  of  tli?  utility  curves  of  systems  with  cotlebooks 
of  40  and  400  dimensions. 

design  is  not  critical,  i.e.,  when  the  designer  can  afford  to  go  lines  and  for  c/j  =  10  0  as  dot-dash  lines.  Only  two  para* 

to  much  lower  bit  density,  it  is  possible  to  enlarge  the  code-  meters  are  used  for  the  code  length  :  Z  =  40  and  Z  =  400. 

book  by  increasing  the  length  of  the  codebook  but  keeping  A  fifth  curve  is  added  to  show  how  a  different  value  for  j3„ 

the  number  of  codebook  words  constant.  In  this  case  one  will  affect  the  curve.  The  dashed  curve  is  plotted  for 

moves  on  &  solid  line  to  the  left.  For  example,  when  going  fic  =  2.0,  a  rather  comfortable  signal  base  for  one  dimension 

from  P  on  the  line  n  10  to  the  left,  one  can  see  that  increas-  (chip).  Note  that  an  increase  of  /Je  shifts  the  curve  merely 

ing  the  length  of  I  from  40  to  100  can  cause  a  gain  in  utility  parallel  to  itself  to  the  left,  exactly  for  the  amount  in  decibels 

of  about  0.7  db.  The  bit  density  is  then  reduced  to  —7  db.  that  /3r  increased  (0  db  in  this  case). 

A  square  codebook  will  be  reached  at  about  —18  db  bit  Figure  22  shows  some  interesting  characteristics  for  systems 

density.  Here  an  orthogonal  codebook  may  be  used  with  a  with  suborthogonal  block  coding.  At  the  left  end  of  each 

utility  of  -f  4.2  db  and  a  SNR  of  — 13.8  db.  Notice,  however,  curve  the  codebook  has  only  two  code  words  (n  1).  Notice 

that  the  rajodel  of  fig.  20  needs  only  —3.5  db  utility  for  the  that  these  points  are  far  below  the  utility  lines  of  the  ideal 

same  bit  density  of  —18  db.  The  difference  is  explainable  binary  system,  though  the  choice  of  two  antipodal  binary 

by  the  fact!  that  fig.  21  is  restricted  to  binary  sequences  and  sequences  as  only  two  waveforms  of  an  ideal  binary  system 

is  an  average  of  all  possible  codes.  would  correspond  completely  to  the  mathematical  model 

The  most  important  results  of  fig.  21  are  tho  maxima  of  equation  34.  But  wo  must  remember  that  the  model 
of  the  dashed  lines.  They  indicate  that,  for  fixed  code  of  fig.  22  represents  the  average  of  all  possible  codes  and  that 

length  1  and  for  given  error  probability,  there  Is  a  maximum  there  are  many  other  pairs  of  binary  sequences  besides 

utility  tha  will  be  reached  at  a  redundancy  between  1  /4  the  optimal  antipodal  sequences. 

and  1/2.  This  maximum  of  the  utility  is  several  decibels  As  one  progresses  to  the  right  on  any  curve  of  fig.  22, 

higher  than  the  utility  that  can  be  reached  by  an  orthogonal  one  deals  with  codebooks  of  tho  same  length  1  and  of  increasing 

codebook  at  a  similar  bit  density.  It  offers  operating  points  numbers  of  code  words.  When  2"  Z,  one  reaches  the 

at  a  much  higher  bit  density  than  the  operating  points  for  point  where  orthogonal  binary  sequences  may  be  used 

orthogonal !  cotlebooks,  when  making  the  comparison  for  (square  codebooks).  Again  it  should  be  noted  that  codebooks 

constant  utility  and  constant  error  ratio.  To  express  these  with  orthogonal  sequences  are  a  very  special  case  of  square 

facts  still  better  we  derive  the  equations  for  the  utility  codebooks  and  it  is  to  be  expected  that  the  average  square 

curves  for  I  codebooks  of  constant  length.  In  analogy  to  codebook  will  offer  a  lower  utility  than  the  optimal  ortho¬ 
equations  63  and  64  we  can  write  :  gonal  eodebooks  can  offer. 

„  „  Increasing  n  over  the  value  log.,  Z  leads  to  suborthogonal 

v,  = _ — _  (05)  codebooks  and  to  a  decreasing  redundancy.  The  eodebooks 

1  . 9 1'  -  fi'D  _  j.  now  become  very  high  rectangles.  At  the  points  of  maximum 

utility  one  has  eodebooks  with  2" 2  and  more  code  words. 
We  recognize  that  to  reach  the  attractively  high  utility  of 
u,it  .  _  io  lOgl0  [—in  (2f‘  ~P'-n  —  ])]  (05a)  -2.2  db,  the  codebook  would  need  21IMI  >  1010  different 

code  words,  even  though  tho  code  length  of  400  binary 
X  _  elements  would  be  manageable.  Beyond  the  maximum 

kvitli  L'  =  y  (log  c«  +  1)  •  1  (66)  of  utility,  the  number  of  code  words  increases  further  and 

fast  approaches  the  maximum  number  of  2'  code  words  of  a 
The  curves  are  plotted  in  fig.  22  for  e«  10-3  as  solid  complete  codebook.  Before  this  ultimate  binary  rate  can 
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bo  reached,  the  curves  turn  towards  u'"'  —  co.  This 

is  the  limit  whore,  in  equation  61,  R.v  :  R.  ,  the  exponential 
bound  parameter.  The  Following  equations  give  the  limiting 
values  directly  : 

1  _  1 

D  <  — -  (log,,  e/j  +  1)  +  —  (67) 

Pr  «  PC 

Letting  l,  the  codebook  length,  grow  beyond  all  limits 
pushes  this  limit  for  D  to  the  value  —pr,lb-  If  Pc  is  at  the 
Nyquist  rate  (pe  0.5;  pc'"’  —  —  3  db).  we  see  that  the 
D  limit  moves  also  to  the  Nyquist  rate  (D'"’  =  3  db).  If 
Pc  2,  the  D  limit  must  be  left  of  the  —3  db  line. 

Corresponding  to  the  D  limit  the  limit  for  n  -if  l  and  5/j 
are  given — is  : 

n  <  log,  on  +  1  +  /  (68) 

There  is  also  an  upper  limit  for  the  utility  of  multidimen¬ 
sional  systems  with  binary  sequences  as  codebook  words. 
This  limit  is  derived  on  page  290  in  Wozencraft  and  Jacobs 
(65)  from  the  exponential  bound.  It  is  given  by  : 

u  <  - = -  or  u'11'  <  —1.41  db  (69) 

2  In  2  1.39 

Kurz  (61)  developed  a  theory  of  efficient  codes,  including 
minimax,  equal  separation,  and  nearly  equal  separation 
codes.  The  coties  are  formed  from  weighted  sums  of  eigen¬ 
functions  generated  by  an  integral  equation  with  its  kernel 
corresponding  to  the  inverse  Fourier  transform  of  the 
Gaussian  noise  spectrum.  This  theory  accomplished  the 
extension  of  a  general  mathematical  model  of  multidimen¬ 
sional  nonbinary  systems  to  channels  with  nonwhite  Gaussian 
noise.  Wolfowitz  (60)  initiated  the  concept  of  a  channel 
where  neither  the  sender  nor  the  receiver  knows  the  channel 
probability  function,  i.o.,  the  error  probability  for  given 
messages.  Based  on  this  idea,  Wolfowitz  provided  in  1961 
“  an  easy  introduction  to  the  ideas  and  principal  known 
theorems  of  a  certain  body  of  coding  theory  for  mathemati¬ 
cians  of  some  maturity  ”  (Wolfowitz  61).  Though  the  author 
claims  in  the  preface  that  “  the  book  has  no  pretension  to 
exhaustiveness,  and,  indeed,  no  pretensions  at  all  ”,  any 
serious  student  of  the  evolution  of  information  theory  in 
the  1950’s  can  use  this  book  as  a  sheer  inexhaustible  source 
of  advanced  proofs  and  theorems,  enabling  him  to  put  the 
engineering  ideas  of  the  1960’s  on  a  solid  theoretical  basis. 

Another  book  by  Gallager  (63)  concentrates  on  the  search 
for  efficient,  binary  codebooks  by  consolidating  the  research 
of  the  1950’s  in  this  area.  Gallager  deals  with  a  special 
class  of  parity  cheek  codes,  called  low-density  parity-check 
codes.  These  codes  are  characterized  by  eodebook  matrices 
containing  mostly  zeros  and  relatively  few  ones.  Chapter  5 
of  the  book  extends  the  major  results  of  the  other  chapters 
to  nonbinary,  low-density  parity-check  codes.  Beyond  the 
treatment  of  these  special  low-density  parity-check  codes, 
Gallager  (65)  offers  a  simple  derivation  of  the  coding  theorem 
that  gave  us  the  exponential  bound  of  the  error  probability 
in  the  form  that  we  used  for  figs.  21  and  22. 

The  optimality  of  the  simplex  code  and  several  other  codes 
in  achieving,  over  a  Gaussian  channel,  the  absolutely  highest 
utility  (if  bandwidth  requirements  are  of  no  concern)  was 
put  on  a  solid  theoretical  basis  by  Landau  and  Slepian  (66). 
The  previously  mentioned  summary  report  by  Gibson  (66) 
may  be  welcome  to  those  readers  who  are  familiar  with  the 
communications  efficiency  as  defined  in  Sanders  (60a). 
Gibson’s  summary  compares  multidimensional  nonbinary 
communications  systems  with  one-dimensional  nonbinary 
(M-arv)  systems. 

K4.  Mathematical  Models  of  Systems  with  Codebooks  Generated 
from  Cyclic  Sequences  ( Shift  Register  Sequences) 

We  shall  discuss  the  experimental  communications  systems 
that  operate  with  binary  shift  register  sequences  in  section  L. 


Most  of  the  theoretical  work  in  this  area  is  still  based  on  the 
original  contributions  of  Heed  (54),  describing  the  famous 
Reed-Mullor  codes,  and  on  the  two  original  investigations 
of  shift  register  sequences  by  Zierler  (55)  and  Golomb  (55). 
A  later  book  (Golomb  64)  gives  some  more  recent  results. 
Basically  all  the  models  resulting  from  these  sources  are 
well  in  iine  with  the  general  models  discussed  above.  Shift 
register  sequences  and,  in  particular,  the  so-called  pseudo¬ 
noise  (PN)  or  pseudorandom  (PR)  sequences  are  also  used 
in  asynchronous  multiplex  systems. 


K5.  Mathematical  Models  of  Systems  with  Biliary  Orthogonal 
Codebooks 


Equation  56  was  used  as  a  convenient  mathematical 
model  for  any  discrete  data  transmission  system  that  uses  a 
set  of  M  equally  likely  equal  energy  orthogonal  signals. 
Equation  58,  a  further  simplification  of  equation  56,  has 
been  plotted  in  fig.  20  over  a  wide  range  of  bit  densities 
and  for  many  error  probabilities. 

In  this  subsection  we  shall  discuss  more  precise  mathe¬ 
matical  models,  many  of  which  have  been  derived  for 
specialized  channel  conditions.  We  intend  to  follow  the 
historical  evolution,  though  we  take  the  liberty  of  discussing 
the  works  of  one  and  the  same  author  out  of  historical  order. 

The  oldest  mathematical  model,  as  far  as  this  author  could 
establish,  is  contained  in  Kotel’nikov  (59a).  originally  pub¬ 
lished  in  1947.  With  reference  to  the  English  edition, 
we  notice  equation  5-12  in  Kotel’nikov’s  notation  : 


P(A,  =  correct)  =  — !— J  [l  -  +  y)]  exp(T)dy 

V-Mt 


In  this  equation  Kotol’nikov  expresses  the  probability  of 
correct  reception  if  A,  (1.)  is  being  transmitted  and  the 
receiver  makes  a  maximum  likelihood  decision.  A,  (t) 
is  one  waveform  out  of  an  orthogonal  set  of  M  equally 
probable  equal  energy  waveforms.  All  waveforms  are 
defined  over  a  time  interval  of  T  seconds  and  are  zero  outside 
this  interval.  Accordingly  P  is  the  probability  of  correct 
reception  for  a  communications  system  operating  with  an 
orthogonal  set  of  waveforms.  Kotel’nikov  uses  in  his  model 
the  following  notations  : 


p.  1 1  :  V  =  --  |  *  o dz  =  Q  (x) 

(See  equat  ion  27  in  this  paper.) 

r+r/i 

p.  30  :  Qs  =  A,  (t)  dt  =  Eo 

7  energy  of  one  waveform  (digit). 

pp.  10-15  :  o-  =  2N„  N„  .  .  .  noise  power  density. 

P  =  1  —  on  on  .  .  .  avorago  error  probability 

per  waveform. 


z  is  a  dummy  variable  for  the  integration  process. 

(The  page  numbers  and  the  notations  at  the  left  side  of  the 
ident  ity  sign  (=)  in  the  above  equations  refer  to  Kotel’nikov’s 
book.) 

Many  other  authors,  independent  of  Kotel’nikov,  arrived 
at  the  same  expression  for  the  probability  of  a  correct 
decision  in  an  orthogonal  system.  We  mentioned  Nuttall’s 
(62)  results  in  connection  with  our  equation  54,  which  for 
A  0  is  identical  with  Kotel’nikov’s  expression.  Reiger  (57) 
published  an  equation  that  proves  to  be  identical  with 
Kotel’nikov’s  expression  when  applying  a  small  substitution 
of  the  dummy  integration  variable.  Vitcrbi,  who  derived 
the  same  equation  in  various  publications  since  1960,  gives  a 
table  of  the  error  probabilities  (Viterbi,  64b,  Appendix  4) 
that  can  be  calculated  from  Kotel’nikov’s  expression. 

When  transforming  Kotel’nikov’s  model  to  the  definitions 
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of  the  utili  y  chart,  one  must  establish  a  relationship  between 
the  digital  error  probability  Cj>  and  the  binary  error  proba¬ 
bility  e n  •  for  the  curves  of  fig.  20,  we  used  equation  57 
for  this  relation.  A  more  precise  relationship  has  been 
derived  by  Viterbi  (00)  and  by  other  authors  (equation  70)  : 


ea 


M 

2<M  -  1) 


eo 


2  n-i 
2”  -  1 


od 


(70) 


M  =  2"  is  the  number  of  waveforms  participating  in  the 
orthogonal  monosignal  system  that  carries  n  bits  of  informa¬ 
tion  in  each  transmission  interval. 

Before  expressing  Kotel’nikov’s  equation  in  the  magnitudes 
of  the  utility  chart,  one  must  take  the  energy  with  reference 
to  one  information  bit ;  i.e.,  E«  =  nEc  .  This  leads  to  the 
implicit  equation  for  the  utility  u  =  N„/E/i : 


(71) 


From  equation  27  it  is  shown  that : 


1  4  Q(x)  =  P(x);  P{x)  = 


V2: 


C‘  ‘~,i  d 7. 


With  the  help  of  this  relation,  one  can  express  the  integral 
in  equation  71  in  the  form  of  equat  ion  54  and  use  the  tables 
in  one  of  the  references  quoted  in  subsection  J7.  Viterbi 
(64,  Appendix  4)  has  calculated  en  and  c n  as  a  function  of 
STb/N0  =  1/u  with  n  =  1,  2,  3,  4,  5,  6,  7,  8,  0,  10,  15,  and 
20  as  parameters.  Some  of  these  curves  are  shown  in  fig.  2 
of  Part  I  of  this  paper  (in  decibel  scale). 

When  plotting  a  utility  chart  one  requires  a  second  equation 
that  relates  the  parameter  n  (bits  per  digit  or  word)  to  the 
bit  density  D.  In  fig.  2  we  used  a  sliding  relationship 
between  the  binary  error  ratio  en  and  the  signal  base  per 
dimension  (chips  or  binary  sequence  element)  p  c  ■  In  figs.  20, 
21,  and  22  we  used  a  constant  value  of  pc  =  0.5  (Nyquist 
rate)  for  all  curves  except  one  for  which  fic  —  2.0  had  been 
selected.  Likewise  in  section  8.5  of  his  1966  book,  Viterbi, 
in  line  with  the  tradition  of  the  codebook  school,  recommends 
the  use  of  the  minimum  signal  base  fic  =  0.5.  However, 
he  points  out  the  fallacies  of  the  use  of  sin  x  over  x  waveforms 
that  had  been  recommended  by  other  authors  as  a  model 
for  the  signal  shape  when  postulat  ing  a  signal  base  of  )3  c  =  0.5. 
i  Instead,  Viterbi  recommends  rectangular  envelopes  of  sine 
and  cosine  functions.  He  defines  as  bandwidth  not  the 
3-db  bandwidth  or  any  other  limitation  of  the  essential 
width  of  the  power  spectrum  but  the  “  minimum  frequency 
separation  between  channels  such  that  signals  of  any  channel 
have  no  effect  on  the  decoders  of  the  others  ”.  To  demon¬ 
strate  the  practicality  of  this  idea,  Viterbi  describes  the 
conceptual  model  of  a  “  coded  phase-coherent  communica¬ 
tions  system  ”  that  uses  orthogonal  binary  sequences  as 
orthogonal  signal  sets  with  each  binary  chip  being  repre¬ 
sented  by  one  of  two  phase  states  of  a  sinusoidal  carrier 
frequency.  The  two  states  have  a  phase  difference  of  180 
degrees  (Viterbi  60a,  b).  Thus  we  see  that  the  waveform 
library  degenerates  in  the  coded  phase-coherent  communica¬ 
tion  system  to  a  binary  biphase  alphabet  (section  H).  If 
one  would  really  build  a  system  of  this  kind  with  many 
densely  packed  subcarriers  so  that  Viterbi’s  definition  of 
bandwidth  could  be  realized,  one  would  arrive  at  a  polysignal 
system  similar  to  the  well-known  Kineplex  data  transmission 
system. 

Accepting  the  pc  —  0.5  value  for  the  sake  of  defining  the 
most  efficient  ideal  model  that  can  be  defined  for  binary  chips 
without  mutual  interference,  we  arrive  at  the  second  equation 
for  systems  with  orthogonal  binary  sequences  : 

I  =  Rb  =  RbTc  =  RaTc  =  _n_ 

B  pc  o.;;  2"-1 


This  equation  is  identical  with  equation  59a,  which  is  written 
in  decibel  notation.  Any  curve  u  =  f(D)  for  constant  e« 
and  for  n  as  the  running  parameter,  when  calculated  with 
the  help  of  equations  71  and  72,  can  therefore  be  directly 
plotted  into  charts  20,  21,  or  22  and  can  be  compared  with 
the  other  mathematical  models  contained  in  those  charts. 
When  doing  so  one  will  find  that  the  Kotel’nikov-Viterbi 
model  is  more  accurate  and  arrives  at  higher  utilities  than 
the  oversimplified  model  of  equation  58a  and  fig.  20.  The 
model  of  fig.  20  shows  the  slow  gain  in  utility  even  when  going 
to  such  extreme  values  as  n  =50  und  en  =  10- 2  °,  values  that 
are  not  covered  by  Viterbi’s  charts  or  tables. 

Kotel’nikov’s  model  of  1947  and  Viterbi’s  subsequent 
implementation  with  the  help  of  binary  sequences  are  not  the 
only  attempts  to  use  orthogonal  signal  sets  for  the  purpose 
of  maximizing  the  system  utility.  As  early  as  1949  Golay 
proved  that  an  extremely  simple  model  of  an  orthogonal 
system  can  reach  the  highest  possible  utility  for  the  Gaussian 
channel.  Golay  (49b)  showed  that,  in  a  system  that  trans¬ 
mits  in  one-out-of-M  timeslots,  one  pulse  of  energy  E/j  can 
reach  the  upper  limit  of  the  utility  u  =  (log,  2)_l  =  1.441 
(or  1 .59  db)  when  M  goes  toward  infinity.  A  system  of  this 
kind  is  called  a  quantized  pulse  position  modulation  system 
(QPPM)  or  a  multiple  time  shift  keying  system  (MTSK). 
Systems  of  this  kind  will  be  mentioned  in  section  L3. 

Similarly  it  was  shown  by  Turin  (59)  that  the  Kotel’nikov 
model  in  Reiger’s  form  (57)  can  be  used  to  submit  the  general 
proof  that  all  orthogonal  monosignal  systems  ultimately 
reach  the  upper  limit  of  the  utility  when  M  grows  beyond 
all  limits. 

Continuing  in  the  chronological  discussion  of  mathematical 
models  for  orthogonal  systems,  we  find  that  Harmuth 
published  in  1960  three  papers  with  detailed  analyses  of 
the  transmission  of  information  by  orthogonal  time  functions 
(Harmuth  60a,  b,  c).  Two  different  kinds  of  orthogonal  sets 
are  discussed  : 

(i)  sets  consisting  of  binary  words 

(ii)  sets  consisting  of  trigonometric  waveforms 

The  first  class  of  sets  uses  rectangular  signals  in  the  time 
domain  and  displays  sin  x  over  x  frequency  spectra  in  a 
low-pass  channel.  The  second  class  of  signals  uses  rectangular 
envelopes  on  sinusoidal  subcarriers.  When  used  in  a  low- 
pass  channel,  the  spectrum  of  such  waveforms  is  much 
narrower  than  the  channel  bandwidth  and  is  centred  at  the 
frequency  of  the  particular  subcarrier.  These  spectra  also 
have  sin  x  over  x  shape.  When  such  signals  of  the  second 
class  arc?  used  in  a  rudio  channel  (bandpass  channel),  the 
subcarrier  frequencies  may  be  directly  placed  into  the  carrier 
band  by  single  sideband  modulation,  or  the  subcarrier 
frequencies  may  be  formed  in  the  low-pass  channel  and  go 
by  double  sideband  modvdation  into  the  carrier  band.  In 
this  case  they  form  a  symmetrical  spectrum  at  both  sides 
of  the  radio  carrier  frequency,  and  the  total  bandwidth 
will  be  doubled.  In  all  the  cases  mentioned  above,  there  is  a 
small  amount  of  cross  talk  generated  between  the  various 
waveforms  when  the  channel  must  be  exactly  bandlimited. 
The  amount  of  such  cross  talk  is  computed  in  Harmuth  (60b). 
The  error  probability  of  the  systems  is  discussed  in  all  the 
papers,  but  in  one  paper  (Harmuth  60c)  an  explicit  formula 
is  derived  for  the  error  probability  when  the  number  of 
orthogonal  functions  is  very  large.  A  special  codebook  of 
32  binary  orthogonal  code  words  is  disclosed  (Harmuth  60c) 
and  a  special  encoding  and  decoding  procedure  by  matrix 
multiplication  is  explained  (Harmuth  60). 

Posner  (62)  succeeded  in  investigating  the  performance 
of  orthogonal  systems  with  correlation  detection  at  very 
low  SNR.  Ho  uses  a  mathematical  approximation  to  equa¬ 
tion  71  that  is  valid  for  small  SNR  and  that  may  be  of 
interest,  to  some  readers.  With  the  help  of  this  approxima¬ 
tion,  Posner  showed  that  at  “  output  SNR  of  less  than 
one  ”  (corresponding  to  utilities  in  the  positive  decibel  range) 
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orthogonal  codes  may  be  less  efficient  than  no  coding  at 
all.  So  far  this  author  has  had  no  opportunity  to  convert 
Posner’s  results  to  the  notations  of  the  utility  chart,  but  the 
reader  is  invited  to  verify  that  the  ideal  binary  system 
(equation  34)  has  a  utility  of  +  0.87  rib  (u  1.22)  for  a 
bit  error  ratio  of  eB  10' 1  while  Viterbi’s  tables  (Viterbi  64b, 
Appendix  4)  indicate  that  an  orthogonal  system  needs  at 
least  n  4  (K  =  4  in  Viterbi’s  tables)  to  reach  e/j  10- 1 
for  u  1.22  (ST/j/No  ==  0.82  in  Viterbi’s  table).  This 

means  that,  for  an  error  ratio  of  0.1,  orthogonal  codes  seem 
to  bo  inferior  to  a  simple  binary  system  (both  operating 
with  correlation  detection)  when  the  code  length  1  is  smaller 
t  han  16  chips.  At  such  large  error  ratios,  it  is  more  profitable 
to  use  two  antipodal  groups  of  4  chips  as  binary  biorthogonal 
waveforms  than  to  use  a  codebook  of  16  chips  and  16  code 
words.  Apparently,  for  utilities  larger  than  +0.87  db, 
the  situation  will  be  much  more  in  favour  of  uncoded  trans¬ 
mission.  Posner’s  investigation  produced  four  theorems 
that  may  prove  helpful  in  any  research  for  transmission  of 
digital  information  in  channels  with  extremely  small  energy 
contrast  (SNR). 

All  the  models  discussed  so  far  assumed  a  perfect  cross¬ 
correlator,  completely  coherent  reception,  and  perfect 
synchronization.  These  are  rather  ideal  assumptions ; 
practical  systems  operating  over  channels  with  fading,  or 
between  moving  stations  are  not  able  to  meet  these  assump¬ 
tions.  Less  ideal  models  were  therefore  required  and  t  he  first 
stop  towards  more  realistic  mathematical  models  were  those 
that  assume  no  phase  coherence.  Ward  (62)  submitted  a  for¬ 
mula  for  the  error  probability  of  a  receiving  system  with  1 
independent  channels,  where  only  one  signal  is  expected  in  one 
of  the  channels  and  t  he  channel  with  the  signal  has  to  be  de¬ 
tected  by  a  “  hugest  of  ”  selection.  The  signal  is  immersed  in 
Gaussian  noise,  all  channels  are  assumed  to  have  the  same 
noise  power  density  N„  ,  and  all  channels  employ  envelope 
detectors.  This  model  can  correspond  to  a  very  simple  ortho¬ 
gonal  system  where  either  1  narrow  band  channels  are  available 
and  a  signal  of  energy  En  is  transmitted  in  one  of  them 
(MFSK),  or  where  1  time  slots  are  available  and  a  signal 
of  energy  Eo  will  be  transmitted  in  only  one  of  them  (MTSK). 

Dumanian  (63)  expanded  the  line  of  thinking  that  had 
been  expressed  by  Ward  (62)  with  a  formula  for  the  error 
probability  in  the  case  where  the  receiver  uses  a  correlator 
after  the  envelope  detector  and  before  the  “  largest-of  ” 
selection.  Sommer  (63)  had  independently  gained  results 
similar  to  those  reported  by  Ward  (62),  and  Splitt  (63) 
derived  an  expression  for  the  limiting  form  of  the  probability 
of  error  as  the  number  of  channels  or  slots  (M)  approach 
infinity.  This  communication  by  Splitt  asserts  that,  also  in 
the  noncoherent  case  (envelope  detector),  the  upper  limit 
of  t  he  utility  u  (log,  2)- 1  will  be  reached  for  M  ->  oo  (D  — >-  0). 
Teplyakov  (63)  derived  an  approximation  of  Kotel’nikov’s 
integral  expression  (equation  71).  This  approximation  con¬ 
tains  only  the  error  integral  (fig.  8b)  and  no  further  integra¬ 
tion  pi'ocess.  It  is  valid  for  very  large  values  of  M. 

Lindsey  (65a)  arrives  at  the  most  universal  model  for 
systems  with  sets  of  orthogonal  signals.  Lindsey’s  model 
can  be  applied  to  the  noncoherent  case,  to  the  partially 
coherent  ease,  or  to  the  coherent  ease.  The  latter  specializa¬ 
tion  leads  to  the  earlier  results  of  Kotel’nikov  (59a),  Reiger 
(57),  Turin  (59),  and  Nuttall  (62).  Lindsey  (67)  expanded 
his  results  to  the  polysignal  case,  where  several  independently 
phase-modulated  carriers  go  over  the  same  radio  carriers. 

Biorthogonal  systems  are  a  simple  extension  of  the  ortho¬ 
gonal  systems  when  applied  in  connection  with  binary 
sequences.  They  use  a  rectangular  codehook  of  1  chips  but  21 
words.  Half  the  number  of  words  is  a  normal  orthogonal 
codebook  ;  the  words  in  the  second  half  are  exactly  the 
negative  of  the  words  in  the  first  half.  Thus  each  word  in  a 
biorthogonal  codebook  has  one  antipodal  word  and  21-2 
orthogonal  words.  The  idea  to  extend  an  orthogonal  set 
to  a  biorthogonal  set  is  so  evident  that  it  is  difficult  to  credit 
any  one  of  the  many  research  workers  in  this  area  with  the 


origination  of  biorthogonal  systems.  In  the  extreme  case 
of  1  1,  a  biorthogonal  system  with  correlation  detection 

becomes  identical  with  the  ideal  binary  system. 

The  first  biorthogonal  codebooks  are  due  to  Muller  (53) 
and  Reed  (54)  ;  a  biorthogonal  system  was  derived  and 
subjected  to  computer  evaluation  by  Viterbi  in  1960.  Also 
Sanders  (59a)  and  Harmuth  (60c)  independently  proposed  the 
application  of  biorthogonal  codebooks  in  data  transmission 
systems.  A  very  general  proof  of  the  optimality  of  biorth¬ 
ogonal  codebooks  is  due  to  Balakrishnan  and  Taber  (63). 

Strong  and  Saliga  (65,  66)  have  recently  reviewed  mos® 
of  the  above-mentioned  mathemat  ical  models.  They  demon¬ 
strate  in  a  number  of  charts  and  tables,  that  for  largo  code¬ 
books  (n  >  6),  phase-coherent  orthogonal  systems  have  a 
better  than  3-db  advantage  in  utility  over  noncoded  systems. 
This  advantage  increases  to  more  than  4  db  when  n  >  10 
(M  21"  =  1024).  The  utility  advantage  of  a  biorthogonal 
system  is  practically  the  same  as  the  one  for  an  orthogonal 
system  for  all  n  >  3.  At  n  =  3  and  below,  biorthogonal 
systems  offer  a  slightly  higher  utility  than  phase-coherent 
orthogonal  systems.  At  n  =  2  and  n  1,  the  biorthogonal 
system  is  identical  to  an  uncoded  system,  indicating  that  any 
kind  of  orthogonal  coding  is  inefficient  at  short  codebook 
lengths. 

When  comparing  phase-coherent  orthogonal  systems  with 
non-phase-coherent  orthogonal  systems  (such  as  MFSK), 
Strong  and  Saliga  show  that,  for  large  n  values,  the  utility 
of  the  non-phase-coherent  system  is  only  less  than  one  decibel 
below  the  utility  of  the  phase-coherent  system.  This  ad¬ 
vantage  of  the  phase-coherent  system  increases  to  about  2  db 
at  n  3  and  at  a  word  error  probability  of  0. 1  (corresponding 
to  eo  —  0.057,  according  to  equation  70).  But  notice  that  at 
high  word  error  probabilities  the  disadvantage  is  always 
smaller  than  at  lower  error  probabilities,  no  matter  what  n 
value  has  been  selected. 

Strong  and  Saliga  do  not  discuss  the  bandwidth  require¬ 
ments  in  detail  but  they  note  that  the  biorthogonal  system 
needs  only  half  the  bandwidth  of  the  phase-coherent  ortho¬ 
gonal  system.  Also  interesting  in  their  charts  is  the  fact 
that  the  uncoded  system  seems  to  have  higher  utility  at 
very  low  error  probabilities  (corresponding  to  the  highest 
utility  values  or  to  the  lowest  SNR  values  in  units  of  E/N0  = 
energy  contrast).  This  would  support  the  calculations  of 
Posner  (62)  that  were  mentioned  above.  Unfortunately 
Strong  and  Saliga  plot  their  curves  only  for  e»  <  10-1. 
For  n  10  this  corresponds  to  e/i  <  0.05  ;  thus  they  avoid 
t  he  range  for  which  Posner  proved  the  superiority  of  uncodod 
transmission. 

C.  L.  Weber  (66)  continued  Balakrishnan ’s  research  (65) 
and  demonstrated  that,  for  a  given  SNR  (utility  in  the 
terminology  of  this  paper)  and  for  a  given  allowed  bandwidth 
(bit  density),  the  problem  of  finding  the  optimum  set  of 
signals  can  bo  reduced  to  the  problem  of  finding  a  non¬ 
negative  definite  matrix  that  maximizes  the  probability 
of  detection.  For  a  rectangular  codebook  matrix  with 
M  =  21,  the  biorthogonal  signal  structure  achieves  the  maxi¬ 
mum  probability  of  detection  at  large  SNR.  The  same 
author  develops  with  R.  A.  Scholtz  as  co-author  (Weber 
and  Scholtz  66)  a  further  extension  of  this  theory  to  the  case 
of  non-phase-coherent  communications,  showing  that  the 
orthogonal  sot  is  yielding  the  lowest  probability  of  error  in 
the  class  of  all  admissiblo  signal  sets  and  that  this  result 
is  true  for  all  signal-to-noise  ratios.  This  result  could  bo 
interpreted  as  indicating  a  contradiction  of  Posner’s  results, 
who  proved  that  uncoded  transmission  is  superior  to  coded 
transmission  at  low  SNR.  Notice,  however,  that  Posner 
compares  the  systems  for  phase-coherent  reception  while 
Scholtz  and  Weber  speak  exclusively  of  non-phase-coherent 
communications.  Also,  Posner  searches  for  the  system  with 
the  highest  utility  for  a  given  error  probability,  while  Scholtz 
and  Weber  search  for  the  system  with  the  lowest  error 
probability  for  a  given  utility. 
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K(S.  M  *  it  h  emu!  teal  Models  of  Systems  with  Suborthogonal 
Codebooks 

Suborthogonal  codebooks  are  those  where  some  or  all 
pairs  of code  words  have  a  positive  cross-correlation  coefficient. 
Even  under  noise-free  conditions,  this  will  cause  some 
positive  output  from  correlators  of  waveforms  that  have  not 
been  transmitted.  If  the  noise  power  is  correspondingly 
I  smaller  than  in  the  orthogonal  case,  correct  MLD  decisions 
|  are  nevertheless  possible  with  the  same  probability  as  in 
I  the  orthogonal  case.  Systems  with  suborthogonal  codebooks 
are  therefore  useful  where  operation  with  lower  utility  but 
higher  bit  density  is  the  goal.  Fig.  21  shows  the  area  of 
suborthogonal  systems  for  the  very  general  mathematical 
model  of  equation  03.  We  recall  that  said  model  applies  to 
the  average!  of  all  conceivable  binary  codebooks  of  the  given 
ratio  2 "/l. 

Helstrom  (55)  derived  the  famous  equation  for  ideal 
binary  systems  (equation  34)  : 

C,>  =  Q{Jl— "} 

A  is  the  cnoss-correlation  coefficient  between  the  two  binary 
waveforms.  |  If  A  =  0,  we  have  the  orthogonal  case ;  for 
A  >  0,  the  system  is  suborthogonal ;  for  A  <  0,  it  is  hyper- 
orthogonal  ;  and,  for  A  =  —  1,  biorthogonal  or  antipodal. 
We  recall  that  this  equation  applies  to  any  pair  of  waveforms  ; 
therefore,  it  is  also  valid  for  any  two  waveforms  consisting 
of  sequences  of  binary  elements.  The  cross-correlation 
coefficient  is  in  this  case  the  ratio  of  the  difference  between 
the  number  of  elements  in  agreement  and  the  number  of 
elements  in!  disagreement  (between  the  two  words)  to  the 
total  number  of  elements.  Such  a  codebook  consisting  of 
only  two  words  can  be  represented  by  a  matrix  of  1  columns 
and  2  rows. 

Slepian  (50)  systematically  investigated  all  binary  code¬ 
books  up  to  1  =  10  as  a  generalization  of  Hamming’s  error- 
correcting  codes.  Slepian  defined  a  class  of  codebooks  that 
he  called  group  alphabets.  This  class  included  Reed-Miiller 
codes  (already  mentioned  as  the  basis  of  biorthogonal  codes) 
and  systematic  codes  as  originally  defined  bv  Hamming  (50). 
Most  of  Slepian’s  binary  signalling  alphabets  are  sub¬ 
orthogonal  dodebooks  and  qualify  completely  as  nonbinary 
systems  when  used  in  connection  with  MLD  decoders. 
Although  Slepian  establishes  important  theorems  for  finding 
“  best  group  alphabets  ”,  i.e.,  suborthogonal  codebooks 
that  maximize  the  probability  of  correct  detection  with  an 
MLD  decoder,  he  does  not  give  an  explicit  or  implicit  formula 
I  relating  tho  error  probability  to  the  SNR  (utility). 

After  research  efforts  had  addressed  the  more  fundamental 
problems  of  finding  orthogonal,  biorthogonal,  and  simplex 
[codebooks,  interest  returned  in  the  last  years  to  systems 
with  suborthogonal  codebooks.  (See,  for  example,  Viterbi  06, 
chapters  8.2  and  8.8,  and  Kurz  01.) 

Hackett  (03)  published  a  theory  for  systems  with  rectangu¬ 
lar  codebooks  (of  binary  elements)  and  compared  the  various 
decoding  methods. 

C.  L.  Weber  (66)  found  “  new  solutions  to  the  signal 
design  problem  for  coherent  channels  ”  by  applying  a 
new  concept  to  the  design  of  codebooks  with  M  >  1.  He 
demonstrated  that  it  is  possible  to  place  K  lower-dimensional 
regular  simplices  mutually  orthogonal  to  one  another  to 
arrive  at  optimal  suborthogonal  codebooks. 

When  continuing  to  search  for  the  mathematical  models 
Jof  more  specific  suborthogonal  codebooks,  we  notice  that 
Bose-Chaudnuri  (BC)  codes  are  used  for  the  most  preferred 
suborthogonal  codebooks  (Gibson  66).  These  codes  were 
first  published  in  1960  (Bose  and  Ray-Chaudhuri  60)  and 
Van  Horn  (61)  developed  a  mathematical  model  for  the 
error  probability  when  using  these  codes  with  bit-by-bit 
decoding  procedures.  The  results  of  Van  Horn’s  investiga¬ 
tion  have  recently  been  compared  in  a  utility  chart  with  the 


general  model  which  uses  the  MLD  procedure  (Filipowsky 
68b).  Some  utility  curves  of  this  MLD  model  are  shown 
in  fig.  22.  The  comparison  in  Filipowsky  (68b)  indicates 
that  BC  codes  should  be  very  attractive  candidates  for  sub¬ 
orthogonal  block  coding  with  MLD.  An  interesting  com¬ 
promise  between  bit-by-bit  decoding  and  MLD  (also  known 
as  probabilistic  decoding)  is  called  “  generalized  minimum 
distance  decoding  ”  and  has  recently  been  introduced  by 
Forney  (66). 

A  number  of  other  suborthogonal  codebook  models  have 
become  known  but  the  respective  publications  do  not  offer 
any  new  mathematical  systems  models. 

K7.  Mathematical  Models  of  Systems  with  Nonbinary  Codebooks 

Shannon  (48)  ;  Feinstein  (54),  (55),  and  (58),  Fano  (52) 
and  (61)  ;  and  Wolfowitz  (57)  are  the  principal  contributors  to 
a  group  of  mathematical  models  of  communications  systems 
that  are  now  generally  known  as  the  exponential  error 
bounds  (WJ  65).  These  error  bounds  are  likewise  applicable 
to  systems  with  binary  codebooks  and  with  nonbinary 
codeboolcs.  When  discussing  the  various  classes  of  binary 
codebooks,  we  used  these  bounds  with  good  results.  For 
nonbinary  codebooks  they  arc  tho  backbone  for  all  mathe¬ 
matical  models. 

Wolfowitz  (61)  gives  on  pages  30-32  a  short  history  of  the 
evolution  of  the  exponential  error  bounds,  stressing  the  fact 
that  this  whole  approach  has  been  “  brilliantly  conjectured  ” 
by  Shannon  in  1948.  In  that  1948  paper  anti  still  more 
explicitly  in  1959,  Shannon  stressed  the  fact  that  the  symbols 
for  the  code  words  (the  letters  in  the  codebook  matrix 
of  fig.  16  in  this  paper)  must  be  taken  from  a  large  alphabet 
(our  waveform  library).  Ideally,  the  size  of  the  alphabet 
should  bo  infinite,  permitting  all  real  numbers  to  be  used 
as  letters  for  the  codebook.  Shannon  and  most  other  authors 
also  invariably  uso  an  MLD  message  (word)  decoder  as  the 
optimum  receiver  when  deriving  their  mathematical  models 
for  nonbinary  codebooks.  Notice  also  that  most  authors 
follow  Shannon’s  example  and  uso  for  their  models  the 
minimum  theoretical  signal  base  /}c  =  0.5  (Nyquist  rate). 
Practical  systems  require  a  larger  signal  base  (time  band¬ 
width  product  per  letter  or  per  dimension  of  the  codebook) 
than  do  theoretical  systems. 

Shannon  (59)  lists  three  possible  limitations  on  the  choieo 
of  code  words  in  a  codebook  with  nonbinary  letters.  These 
limitations  lead  to  three  classes  of  codebooks  that  yield 
quite  similar  results  in  their  overall  performance  but  still 
require  different  equations  for  their  mathematical  models. 
Shannon  also  stresses  the  fact  that  he  deals  in  his  1959 
paper  only  with  the  case  of  average  power  limitation  on  tho 
code  words,  whereas  in  his  earlier  paper  in  1948  he  discussed 
also  the  channel  capacity  with  a  peak  power  limitation.  The 
three  classes  under  average  power  limitation  discussed  in 
the  1959  paper  are  the  following  : 

(1)  All  code  words  are  required  to  have  exactly  the  same 
power  P  or  the  same  distance  from  the  origin  (in  an 
1-dimension  vector  space).  Code  words  are  in  this 
case  represented  by  vectors  with  end  points  lying  on 
the  surface  of  a  hypersphere  of  radius  \  IP. 

(2)  All  code  words  have  power  P  or  less.  In  this  case  all 
code  words  lie  on  the  surface  of  a  hypersphere  or  within 
its  interior. 

(3)  The  average  power  of  all  code  words  is  P  or  less.  That 
means  some  code  words  may  lie  outside  the  sphere 
of  radius  VlP  and  others  may  lie  inside  the  sphere. 
If  all  code  words  are  used  equally  likely,  the  average 
power  will  remain  P  ;  but,  if  some  code  words  have  a 
different  probability  from  others,  the  average  power 
over  a  long  interval  may  differ  from  P  but  never 
should  be  larger  than  P. 
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In  the  following  conversions  of  the  results  of  Shannon 
and  of  other  authors  to  the  notations  of  the  present  paper, 
we  shall  always  assume  codehooks  of  class  (1)  unless  stated 
differently. 

The  power  of  a  codeword  is  defined  as  : 


for  a  code  word  of  1  real  numbers  fti  (fig.  16). 

The  exponential  bounds,  in  their  most  general  form, 
may  be  expressed  in  the  following  form  : 

eK  =  0(1,  D)  .  exp  {-  1  .  E,.  (1,  D)}  (74) 

The  various  forms  of  exponential  bounds  differ  in  the  struc¬ 
ture  of  the  two  functions  C(l,  D)  and  E  /,  (1,  D).  The  function 
C(l,  D)  is  frequently  called  the  “  constant  ”  of  a  particular 
exponential  bound.  It  is  true  that  its  influence  on  the  error 
probability  is  less  significant  than  the  influence  of  E  r,  (1,  D). 
This  second  fimetion  E  /,  (1,  D)  is  normally  called  the  reliability 
function  of  any  exponential  bound.  Indeed,  the  “  constant  ” 
may  be  represented  by  a  constant  value  over  large  ranges 
of  i  and  D.  For  the  sake  of  simplicity  we  shall  normalize 
this  “  constant  ”  to  unity  for  much  of  the  following  presenta¬ 
tions.  The  reader  will  easily  recognize  how  to  introduce 
other  values  of  C(l,  I)),  when  required  for  special  purposes. 
Shannon  (59)  and  Gallager  (65)  offer  many  equations  for 
C(l,  D),  thus  permitting  the  application  of  the  normalized 
exponential  bounds  to  many  special  cases.  Brillouin  (56) 
relates  this  general  exponential  bound  (his  equation  17.16) 
to  the  general  probability  law  of  thermodynamics. 

The  reliability  function  E  i.  (1,  D)  determines  the  general 
behaviour  of  the  mathematical  models  of  codebooks  with 
nonbinary  elements.  Shannon  (59)  offers  a  series  of  charts 
for  Ei  (Rs//)  which  can  be  used  easily  to  derive  utility 
curves  for  mathematical  models  of  nonbinary  codebooks. 
Figure  23  shows  how  to  use  Shannon’s  charts.  At  the  left 
side  there  is  a  typical  set  of  curves  showing  Shannon’s 
E  i.  for  the  case  of  his  parameter  A  2  (taken  from  his  fig.  8), 


corresponding  to  our  a  =  A2  4(SNR  of  +6  db).  At 
the  right  side  of  fig.  23  we  show  where  special  points  of 
Shannon’s  curves  can  be  found  in  the  utility  chart. 

When  converting  Shannon’s  magnitudes  to  ours  one  must 
consider  that  Shannon  uses  natural  logarithmic  units  (nits) 
for  the  information  rate.  Thus  his  rate  parameter  R .?//*  can 
be  related  to  our  bit  density  by  the  following  derivation  : 

R  log,  M  1  log,  M  log,  M  log,  e  (log,  M) 

D  B^  Tip  '  B  =  Tc  B  '  “1 JT  ft  ' 


log,  e 

IT 


■  R SH 


(75) 


Using  the  minimum  signal  base  per  character  (or  dimension) 
of  fic  0.5  (as  Shannon  assumes)  we  arrive  at 

D  -  2(I.442695)R,S//  =  2.885390  .  R,s-//  ,  (76a) 


or  the  reciprocal  relationship 

Rs//  =  0.3405767  D  (76b) 

In  decibels  this  relation  can  bo  written  as 


D'"’  =  4.602045  +  R£‘  (76c) 

With  the  help  of  these  relations  one  can  verify  that 
Shannon’s  chart  indicates  a  maximum  rate  of  Its//  =  0-8 
on  the  R«  scale  for  E/,  =  0  (point  P,).  This  corresponds 
to  a  bit  density  of  2.3083  or  -f-  3.633  db.  This  indeed  is  the 
point  in  the  utility  chart  where  the  a  6  db  line  cuts  curve  1 
(Shannon’s  upper  bound,  discussed  in  Part  I  of  this  paper). 

To  derive  utility  curves  for  nonbinary  codebooks,  one  must 
use  all  the  reliability  curves  of  Shannon’s  1959  paper.  The 
following  procedure  was  used  to  arrive  at  the  curves  of  fig.  24  : 


STEP  1  :  Compute  the  reliability  parameter  from  the 
given  set  of  data  :  &b  ;  1 ;  C(l,  I))  ;  ftc  ■  This  can  be  done 
by  solving  equation  74  for  E/,  and  by  converting  the  word 
error  ratio  e,„  to  the  bit-error  ratio  6b  in  lino  with  the  fre¬ 
quently  used  upper  bound  cb  <  e„/2.  The  result  is  : 


•To  avoid  any  confusion  with  our  transmission  rate.  It.  we  give 
Shannon’s  rate  parameter  the  subscript  SH  standing  for  Shannon. 
Shannon’s  N.s//  corresponds  to  1  in  this  paper. 


Figure  23. — Conversion  of  reliability  parameters  into  utility 
parameters. 
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„  log,  (2eB)  -  log,  C  (1,  D) 

Et  - - j -  ("7) 

This  is  a  constant  value  if  5b  and  1  and  C  are  kept  constant. 
For  the  normalized  presentations  that  follow  we  put  C(l,  D)  s 
1  or  log,  C(lJ  D)  =  0. 

As  an  example  we  may  assume  e«  10-3  and  log,  2co  = 
—  6.2146.  Assume  that  1  =  10  makes  Et  =  0.02140  for 
this  case.  Entering  this  value  in  fig.  23a  leads  to  points 
P5 and  P, . 

STEP  2  :  Use  the  various  charts  of  Shannon’s  paper 
and  determine  for  each  SNR  (u  =  A'-')  the  Ksn  values  for 
upper  bound  and  lower  exponential  bounds.  These  values 
must  be  rcajd  from  the  correct  R  scale.  For  1  =  10  one 
must  use  the  R,„  scale.  The  chart  for  A  =  2  (a  =4, 
it'"'  =  0.02 1  yields  for  E/.  0.02140  the  R $//  values  of 

Rsw  =  0.16  and  Rs//  0.37.  With  the  help  of  equation  76e 
(assuming  /5  0.5)  we  find  the  corresponding  values  of 

D  —  —3.30  db  and  D  =  +0.28  db.  The  points  P5  and  P6 
can  now  bo  pound  as  the  intersection  of  the  ordinates  with 
the  o  =  0.02  db  line.  Any  other  /9C  value  would  shift  the 
D'"'  values  for  3.01  +  jS,'"'  to  the  left. 

STEP  3  :  Continue  to  find  the  D  values  for  all  other  a  lines 
for  which  A  2  =  a  is  larger  than  2E/,  .  It  can  be  easily 
seen  that  the  highest  E  /,  value  on  each  of  Shannon’s  charts 
is  A2/2.  If  any  E/,  value  computed  under  step  one  is  larger 
than  A2/2  for  any  given  chart,  the  chart  can  not  be  used. 
In  our  example  this  is  the  case  for  A  =  1  and  all  smaller  A 
values.  In  terms  of  the  utility  chart  this  means  that  the 
utility  curve'  for  the  upper  bound  of  Shannon’s  model  for 
1  1 0,  e/i  10" 3  is  entirely  to  the  right  of  the  a  -  0  db  line. 

Instead  of  graphically  finding  the  intersection  of  the 
vertical  D  liras  with  the  diagonal  a  lines,  one  may  calculate 
the  ut  ility  from  the  R sh  values  and  the  A  values  of  Shannon’s 
charts  by 

lyo  =  R+  +  4.602045  -  20  log10  A  (78) 

This  process  of  converting  Shannon’s  reliability  charts 
to  our  utility  charts  has  been  completed  for  a  number  of 
1  and  5b  values.  The  results  are  shown  in  fig.  24.  To  avoid 
the  overlapping  of  too  many  curves  we  omitted  the  above 
example  1  10,  5b  =  10~3. 

When  discussing  the  meaning  of  fig.  24  we  recognize  that 
the  curves  for  larger  codebooks  come  closer  to  the  utility 
pf  curve  I,  Shannon’s  upper  bound  of  the  channel  capacity. 
The  difference  between  upper  exponential  bound  and  lower 
exponential  bound  (the  shaded  areas)  is  larger  for  small  1 


values  and  for  small  5s  values.  Both  bounds  becomo  prac¬ 
tically  identical  when  the  codebook  is  large  and  when  a 
high  error  error-probability  can  be  tolerated  (e.g.,  the  case 
1  =  100,  5b  =  10->). 

The  pract  ical  importance  of  upper  bound  and  lower  bound 
is  subject  to  various  interpretations.  One  frequently 
mentioned  interpretation  says  that  no  code  of  length  1  with 
nonbinary  elements  can  be  found  that  would  give  a  better 
error  ratio  than  5 a  when  the  system  operates  with  the  bit 
densities  (number  of  code  words,  M)  and  the  utilities  (recip¬ 
rocal  energy  contrast)  defined  by  the  curves  of  the  upper 
bound.  This  is  the  curve  from  P,  to  Pe  ,  generally  called 
the  sphere-packing  bound  and  originally  derived  by  Elias  (55). 
The  same  interpretation  declares  the  lower  bound  as  limit, 
indicating  that  at  least  one  code  must  exist  that  makes  the 
system  operate  with  a  better  utility  than  the  one  indicated 
by  the  curve.  This  bound  is  actually  computed  from  an 
upper  bound  on  the  average  of  all  codes  in  the  class,  arguing 
that  the  best  code  in  a  class  surely  must  be  better  than 
the  average.  This  bound  is  known  as  the  random  coding 
bound  and  is  also  due  to  Elias  (55)  and  Shannon  (57).  It 
coincides  with  the  upper  bound  from  point  P[  to  P2  in  fig.  23a. 
Then  it  continues  as  a  straight  line  to  Pu  and  on  to  P , ,  . 

From  P„  to  P,  in  fig.  23a  there  is  a  tighter  lower  bound 
called  the  expurgated  random  coding  bound  conceived  by 
Shannon  (57)  and  further  exemplified  by  Gallager  (65). 
The  expurgated  random  coding  bound  reaches  R  «  =  0  at 
E/,  =  A2/4.  This  point  extends  in  fig.  24  into  a  a  line: 
a<"‘  =-  E  \.db  +  0.02.  These  are  all  the  solid  diagonal  lines 
in  the  left  half  of  fig.  24. 

When  using  this  interpretation  it  is  essential  to  remember 
that  the  curves  are  normalized  for  a  coefficient,  C(l,  D)  =  1. 
Small  corrections  will  be  needed  in  special  cases  where  C 
deviates  significantly  from  1.  Notice,  however,  that  in 
equation  77,  (1,  D)  may  vary  in  a  range  from  0.1  to  100 
without  significantly  influencing  E  /.  when  5b  is  very  small. 
Also,  when  1  is  vory  large,  E  /,  becomes  very  small  and  its 
total  influence  on  the  ut  ility  curve  becomes  rather  negligible. 

Any  point  on  the  upper  arm  of  the  upper  bound  in  fig.  23a 
(for  example,  the  point  P,)  requires  a  reliability  that  is 
higher  than  the  highest  reliability  of  the  lower  bound.  This 
means  that  there  is  no  assurance  that  a  code  can  be  found 
for  such  an  operating  point.  It  merely  says  that  no  code 
can  be  found  with  a  still  higher  reliability.  We  therefore 
plotted  this  upper  arm  of  the  upper  bound  in  dashed  lines 
in  fig.  24,  cross-connecting  upper  bound  and  lower  bound 
with  a  solid  line,  the  so-called  Plotkin  bound  (Plotkin  60). 
It  is  based  on  a  bound  on  the  average  distance  between 


Figure  24. — Utility  curves  of  systems  with  linear-real  cotlcbooks. 
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code  words  that  can  be  found  by  adapting  techniques 
originally  applied  to  binary  codes  (Peterson  and  Kasami  65). 
The  Plotkin  bound  is  shown  in  fig.  23a  as  the  straight  lino 
at  E  i.  =  A2/4  between  P  „  and  P10  . 

When  charts  from  Shannon’s  1959  paper  are  used,  it  is 
evident  that  they  become  inaccurate  for  extreme  values 
of  Bi  — >  0  or  A 2  — >  0.  Fortunately,  in  the  same  paper 
Shannon  derived  two  explicit  equations  that  are  accurate 
representations  for  the  lower  bound  and  the  upper  bound 
when  a  is  smaller  than  about  — 10  db.  He  also  gave  another 
equation  that  is  valid  as  a  joint  upper  and  lower  bound  for 
rates  close  to  the  channel  capacity.  We  shall  now  discuss 
these  three  approximations  and  convert  them  to  the  units 
of  the  utility  chart  : 


Approximation  1  :  Lower  bound  for  a  <  0. 1 

This  is  the  random  coding  that  is  represented  in  fig.  23a 
by  the  straight  line  from  P2  to  Pu  and  P,,  .  It  can  be  seen 
from  Shannon’s  reliability  charts  that  the  expurgated 
random  coding  bound  degenerates  into  the  normal  random 
coding  bound  when  the  SNR  a  <4  1  (a  <  —10  db).  Points 
P,  ,  Pn  ,  and  P, 2  join  in  a  single  point.  Shannon  has  plotted 
these  bounds  for  small  A  ( Vo  in  our  notation)  in  a  special 
figure  (fig.  13)  in  his  1959  paper.  Peterson  and  Kasami  (65) 
gave  a  larger  version  of  the  same  curve.  In  these  presenta¬ 
tions  the  lower  bound  follows  the  equation 


u"6  =  -  1.41855  db  +  D'"’  -  10  log,„  (D  +  2.88539  EjJ  (79c) 

,  D 

This  is  the  equation  for  a  utility  curve  when  a  —  <0.1. 

u 

It  can  be  seen  that  this  equation  reaches  maximum  utility 
for  E  l  — >  0.  The  value  is  : 

udb  =— 1.41 855  db  (79d) 

max 

This  value  also  occurred  as  the  limiting  value  for  binary 
codebooks  in  figs.  20  and  21.  Indeed  these  figures  aro 
based  on  the  same  exponential  bound. 

Another  interesting  limiting  case  may  be  derived  from 
equation  79  for  cases  where  D +;  2.885  E  /,  ,  i.e.,  at  the  extreme 
left  side  of  the  utility  chart.  There  we  get 

ud*  ^  Ddb  —  6.0206  db  -  Edb  (79e) 

Considering  that  Dd!>  —  u{lb  =  a’"’,  we  can  write  equa¬ 
tion  79e  also  in  the  form  : 

odb  =  E«  +  6,0206  ,  or  d  =  4Et  (79f) 

This  is  the  limiting  value  for  the  diagonal  a  line  if  D  — >  0 
(Ddb  -  co). 

Approximation  2  :  Upper  bound  for  a  <  0.1 

This  is  the  sphore-packing  bound.  Shannon  gives  an 
approximate  equation  for  the  reliability  of  this  bound  under 
the  assumption  that  A  =  V a  <4  1.  In  Shannon’s  notations 
this  equation  takes  the  form  : 

A2  c2 

Ei,  (R)  =  — - Af  +  — 


The  magnitude  c  is  related  to  Shannon’s  signalling  rate  R 
(again  for  A  1 )  by  the  relation  : 

R  ^  —  or  V2R 
—  2 


Eliminating  e  from  the  approximate  equation  for  E 
leads  to 

E,,  ^  1  ji R  R 
=  2  “  V  A2”  ~r  A2 

Shannon  (59)  in  his  fig.  13  and  Peterson  and  Kasami  (65) 
in  their  fig.  2  give  fairly  accurate  normalized  charts  with 
plots  of  10 1, /A 2  versus  R/A2.  For  many  applications  it 
may  be  desirable  to  solve  this  equation  for  the  efficiency 
parameter  of  this  paper,  the  utility  u.  This  can  be  done 
with  the  help  of  equation  76,  remembering  that  we  added 
to  Shannon’s  R  (used  in  the  equations  above)  the  subscript  SH 
to  distinguish  it  from  the  parameter  R  used  in  this  paper. 
The  result  of  the  conversion  to  the  notations  of  this  paper 
is  the  following  approximate  equation  : 

ntiEr^0^*”-2'733^]  ,80» 

The  reliability  function  Et  ,  which  can  be  determined 
from  equation  77,  will  be  a  constant  for  fixed  en  ,  C,  and  1. 
The  constant  a  is  known  to  any  desired  precision  : 

log,  2 

a  =  — —  =  0.34657359  (80a) 

Equation  80  may  be  expressed  in  the  familiar  decibel 
notation  : 


udb  =  -  3.0103  +  D"  +  10  log10  [E,,  +  aD  -  2  V aDEtj  - 

-  101oglo[(Et  -  aD)2]  (80b) 

The  square  on  the  inner  bracket  of  the  last  term  is  retained 
to  indicate  that  negative  values  of  E  /,  —  xD  which  frequently 
will  occur  do  not  disturb  the  formation  of  the  logarithm. 
One  could  also  write  20  log,„  [±(Ez  —  aD)j  for  this  term. 

Again  one  may  further  simplify  equation  80  when  assuming 
that.  aD  <4  E  i,  .  This  means  one  is  interested  in  the  utility 
curves  of  this  model  at  the  extreme  left  side  of  the  chart 
where  D  — >  0  (D“  — >  —  oo).  In  this  range  the  utility  is 
simply 

udb  =  -  3.0102  +  Ddb  +  Edb  (81) 

or 

odb  =  3.0102  +  Edb  (8  la) 

The  meaning  of  equation  81a  is  that  the  utility  curves 
at  the  extreme  left  of  the  chart  go  into  constant  a  lines 
shifted  for  3  db  to  the  right  against  t  he  line  for  E  Ldb-  Because 
aD  must  remain  much  smaller  than  E  /,  ,  one  can  see  that 
this  simple  equation  will  be  valid  only  for  very  small  D. 
Under  these  circumstances  this  utility  line  corresponds 
to  the  point  Ps  at  E/,  =  A2/2,  except  that  the  relationship 
of  equation  81a  would  apply  only  to  reliability  charts  for 
A  <4  1.  The  utility  line  of  equation  81a  corresponds  also 
to  the  highest  point,  Ei/A!  =  0.5,  of  the  previously  men¬ 
tioned  normalized  charts  of  Shannon  and  Peterson. 

Another  simplification  of  equation  80b  is  possible  when 
assuming  aD  >  E/,,  provided  we  still  operate  in  an  area 
of  tho  utility  chart  where  a  <4  1  (condition  for  the  approxima¬ 
tion  in  80b).  This  means  aD  1.  Thus  we  can  state  that  if 


we  get 


or 


E/,  aD  <4  I 


u  =  —  =  - - -  =  log2  e  =  1.4427 

2a  log,  2 

=  +1.59206  db 

max 


(81b) 


(81c) 
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This  is  tht  highest  utility  that  Shannon’s  1948  bound  for 
ihc  channel  capacity  can  ever  roach  (Part  I)  and  that  only 
for  D— >0.  Wo  can  now  understand  the  moaning  of  the 
restraint  expressed  by  equation  81b.  Yet  it  is  gratifying  to 
see  that  this  explicit  equation  80b  for  the  upper  bound  of  the 
utility  of  codebooks  with  nonbinary  elements  can  roach 
the  top  utility.  The  question  is  therefore  how  small  Ex. 
must  be  made  to  come  close  to  umBX  ;  this,  in  turn,  is  related 
to  the  problem  of  finding  the  trade-off  numbers  between  the 
dimensionality  of  the  codebook  and  the  loss  of  utility  against 

Umax  • 

Approxi  matiin  .'5 :  Bound  near  channel  capacity  (l  J>  1) 

Figure  23  sh  >ws  that,  between  point  P,  and  point  P,  ,  the 
lower  and  upper  bounds  of  the  reliability  (fig.  23a)  and  of 
the  utility  (  ig.  23b)  coincide.  This  characteristic  of  the 
reliability  eutves  is  valid  for  any  A  (or  V  n)  value.  In  the 
Utility  chart,  we  may  therefore  expect  to  meet  this  combined 
pound  along  jany  constant  a  line,  no  matter  if  a  is  small  or 
large.  On  the  other  hand,  it  is  clear  that  this  combined 
pound  will  always  be  the  part  of  a  reliability  curve  (or  a 
Utility  curve)  that  comes  closest  to  Shannon’s  curve  number  1 
representing  the  channel  capacity. 

Shannon  (48),  recognizing  earlier  work  by  Toiler  (49)  and 
Wiener  (48),  defines  (in  bits  per  second)  the  capacity  of  a 
continuous  channel  perturbed  by  white  Gaussian  noise  as 
C  =  W  log,  ( 1  +-  a) 

In  his  1959  paper  Shannon  uses  the  same  channel  capacity 
in  nits  (natural  units  of  information)  per  degree  freedom 
(dimension  ol  the  codebook)  and  ho  assumes  that  215  dimen¬ 
sions  pei-  second  may  be  transmitted.  This  gives  (in  nits 
per  dimension)  a  channel  capacity  (designated  by  this 
author  as  C«jt)  of 

C sn  =  0.5  log,  ( 1  +-  o)  (82) 

In  most  of  Uu'  utility  charts  wo  plotted  this  channel  capa¬ 
city  as  a  utility  curve  (usually  curve  number  1).  It  is 
defined  by  equation  21.  repeated  below,  using  the  sub¬ 
script  “  1  ”  to  identify  these  utility  and  bit  density  values 
ns  those  that  relate  to  Shannon’s  1949  channel  capacity  : 

ju»  D-;*  -  101oglo(2°>  -  1) 

where  D,  is  the  expression  log,  (1  +-  a)  as  may  be  seen  from 
equation  18  in  Part  I.  Therefore  wo  may  easily  relate 
Shannon’s  1959  notation  Csu  to  our  D,  ,  using  st  from 
equation  80a  a 


C su  =  0.5  log,  2  [log,  (1 


log,  2 

°)]  =  -^—  D,  =  ccD, 


Reliability  charts  are  frequently  expressed  in  Shannon's 
1959  notation  using  nils  per  degree  of  freedom  as  the  units 
of  the  channel  capacity.  Equation  83  gives  the  connection. 
T|he  bit  density  D,  ,  corresponding  to  the  channel  capacity, 
may  be  read  off  curve  1  as  the  horizontal  distance  of  any 
point  on  the  curve  from  the  vertical  axis  expressed  in  decibels. 
Converting  this  value  to  numbers  and  multiplying  it  with 
a  gives  the  channel  capacity  C sh  ■  This  has  been  done 
bpforo  in  fig.  23b  for  point  Pj  .  In  that  case, 

IF*  10  log,, ,  |  log,  (1  +-  o)f  =  3.658  db  or  D,  -  2.3215, 

aD,  =  0.804  nits  per  dimension. 

With  this  definition  of  C sn  and  using  Hsn  (as  before), 
tile  reliability  near  channel  capacity  may  be  expressed  as 
follows  (Shannon  59,  equation  74)  : 

(1  +  op 

E/.  =  — — — - -  (C sn  —  Rjs/;)2  (84) 

o(2  +-  °) 

Introducing  the  equations  for  Cs/i  and  R sn  yields  : 

l  (l  +  <d2 

Pi.  =  -WTT-W  r«2  (D,  -  DPI  (84a) 

O  (2  +  a) 


Because  a  =  — ■ ,  one  may  solve  this  equation  for  u,  but 
u 

the  solution  is  a  rather  complex  equation,  since  D,  is  also  a 
function  of  u.  Consequently,  wo  prefer  to  express  the  bit 
density  of  the  approximate  combined  bound  as  a  fraction 
of  tho  channel  capacity  D,  . 

D  =  8D,  ;  D"*  D"‘  +  S''*  (85) 

The  factor  S  is  always  smaller  than  “  1  ”,  but  close  to  “  1  ”. 
In  decibels  it  will  have  small  negative  values.  Introducing  8 
into  equation  84a  and  using  for  D,  =  log,  (1  +-  a)  leads  to 
an  equation  for  8  as  a  function  of  E/,  and  a  : 


8  =  1- 


*v/ 15;.  a  (2  4-  o) 
a(l  +  a)  log,  (l  +  o) 


or  in  decibels  : 


S'1*  10  log 


Ci  VE;.  o(2  +  a) _ -| 

°  L  a(l  -f  o)  log,  (1  +  a)J 


For  any  constant  value  of  o,  one  can  express  the  equation 
in  the  simple  form  : 

8"*  10  1oglo[l  -  d(o)  YeT)  ,  (86b) 

with  the  constant  d(o)  defined  by: 

\/a(2  -{-  o) 

A(o)  =  -  - 5 - - - (86c) 

a(l  +  o)  log,  (1  +-  a) 

and  a  from  equation  80a  has  the  numerical  value  : 


a  —  0.34657 
A  few  characteristic  values  are  : 


oJb 

-10  —0  -3 

0 

+  3  j 

-fG 

+  10 

+  20 

A(o) 

8.053  5.294  3.042 

2.500 

1.980 

1.216 

0.83 1 

0.500 

With  the  help  of  these  values  it  is  easy  to  plot  utility 
curves  near  channel  capacity.  E  can  be  calculated  from 
equation  77  for  any  fixed  e«  and  1.  If  a  particular  mathe¬ 
matical  model  is  to  be  considered  with  C(l,  D)  significantly 
different  from  unity  anywhere  in  the  desired  range  for  D, 
equation  77  still  may  be  used,  but  E;,  will  now  be  a.  function 
of  1  and  D.  Once  E  j,  is  found,  one  draws,  in  a  utility  plot, 
the  diagonal  lines  for  various  a  values ;  for  example,  for 
those  listed  in  the  table  above.  Using  the  precalculated 
A  (a)  values,  or  calculating  them  for  other  a  values  with  the 
help  of  equation  86c,  and  applying  8fib  gives  8  in  decibels. 
This  value  is  the  distance  on  the  horizontal  axis  that  one 
has  to  go  to  the  left  from  D,  (the  intersection  of  the  a  line 
with  the  curve  1)  to  find  the  D  value  of  the  desired  utility 
curve  for  the  particular  o-  value.  The  corresponding  u 
value  results  as  tho  intersection  of  that  vertical  D  line  with 
the  a  line  or  from  the  relationship  u'lb  =  D'"'  —  a'lb.  Thus 
one  can  plot  a  utility  curve  point  by  point  by  assuming 
reasonable  a  values  and  calculating  the  corresponding  D 
and  u  values. 

Figure  25  shows  a  number  of  interesting  utility  curves 
computed  with  the  help  of  theso  three  approximate  formulas. 
The  curves  must  be  computed  for  large  l  values  to  keep  them 
at  the  left  side  below  —10  db  signal  contrast  and  to  keep 
them  on  the  right  side  close  to  channel  capacity.  These 
curves  demonstrate  the  advantages  of  nonbinary  codebooks 
with  sufficiently  long  code  words.  The  number  of  code 
words  that  are  required  to  meet  a  given  bit  density  can  bo 
calculated  from  the  definition  of  bit  density  as 

R  n  /  1  \  log,  M  log,  M 

D-  — =  —  (  —  )=  — —  =  ^— ;  (87) 

B  T„  V  B  /  T,/B  p  J. 

M  =  2^c"'  =  2“  .  (Sec  also  equation  65.)  (87a) 
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Figure  25. — Utility  curves  defined  by  the  approximate  equations  for 
an  ideal  system  with  linear -real  codebooks. 


Figure  25b  shows  the  interesting  section  —  5  <  I)'"'  <  -)-3 
in  a  larger  scale.  In  this  range  the  curves  follow  the  third 
approximation. 

Apparently  M  will  he  unrealistically  largo  for  any  attempt 
to  arrive  at  the  right  side  of  the  utility  chart  close  to  the 
channel  capacity.  The  practical  engineer  is  therefore  most 
interested  in  utility  curves  with  small  1,  so  that  he  can  keep 
the  whole  codebook  in  reasonable  proportions.  Unfor¬ 
tunately,  none  of  the  approximate  formulas  are  valid  for 
small  1.  The  accurate  expressions  for  the  error  probability 
all  depend  on  the  error  function  (probability  density  integral) 
and  thus  can  not  be  expressed  in  the  explicit  form  u  =  f(D). 
Several  authors,  however,  performed  computer  calculations 
and  arrived  at  charts  that  can  he  used  for  graphical  evalua¬ 
tion,  similarly  as  we  used  Shannon’s  charts. 

In  Part  I,  fig.  5,  we  used  Slepian’s  curves  as  an  example 
for  the  utility  curves  of  ideal  nonbinary  systems.  These 


curves  have  been  converted  to  our  notation  from  Slepian  (63). 
In  fig.  5  only  the  upper  bound  is  plotted,  but  this  is  done  for 
two  different  error  ratios.  Now  we  would  like  to  show  in 
fig.  26  the  wide  spread  between  upper  hound  und  lower 
hound  when  1  has  a  small  value.  To  avoid  overcrowding 
the  chart,  wo  do  so  only  for  one  fixed  error  ratio  (Cc  5.10  5). 

Slepian  (63)  has  included  the  reliability  function  and  the 
coefficient  of  the  exponent  ial  hounds  in  all  his  computations. 
Pie  plots  curves  for  a  fixed  error  ratio  exactly  as  we  do. 
When  converting  his  word  error  ratio  to  the  hit  error  ratio 
of  (his  paper,  we  again  use  the  relation  c»  —  2 e«  .  Since 
Slepian  publishes  his  curves  only  for  a  few  fixed  word  error 
rat  ios,  we  used  e n  =  5.10  5  as  the  fixed  value  for  our  fig.  26, 
corresponding  to  his  P,  10  1 .  In  fig.  5  in  Part  I  we 
applied  a  graphical  correction  to  arrive  at  bit  error  ratios 
of  10-*  and  10- *. 

The  vertical  axis  of  Slepian’s  curves  is  the  SNR  difference 


Figure  26. —  Upper  atul  lower  bouiuls  for  the  utility  of  a  system  with  a 
fixed  number  of  dimensions  ( after  Slepian). 


December,  1969 


Proceedings  I.R.E.E.  Australia  437 


FILIPOWSKY  :  Nonbinary  Information  Transmission 


of  an  ideal  system  with  a  nonbinary  codebook  of  n-dimensions 
(1  in  our  notation)  against  Shannon’s  ideal  system  with  an 
infinite  number  of  dimensions  (our  curve  1).  The  units 
of  the  vertical  axis  are  in  decibels  and  we  may  call  this  a 
difference  r. 

The  horizontal  axis  in  Slepian’s  curves  (r)  is  the  bit  density 
as  defined  in  this  paper,  but  the  scale  is  based  on  the  common 
logarithm  in  place  of  the  logarithm  to  the  base  “  2  ”  as  used 
herein.  His  units  appear  therefore  in  dits  per  Hz  bandwidth. 
This  forces  the  engineers  alternately  to  use  bits,  nits,  or 
dits,  depending  on  the  individual  author.  (Wo  hope  the 
unified  presentation  in  this  paper  will  be  appreciated.) 

The  conversion  formulas  from  Slepian’s  notations  to  ours 
are  (SL  standing  for  Slepian)  : 

u"  =  -  10  log10  (2»  -  1)  (88) 

D"*  =  5.2139  +  10  log10rSi.  (89) 

Figure  20;  shows  the  wide  spread  between  upper  and  lower 
bound  at  low  1  values  and  for  bit  densities  around  zero 
decibels.  Slepian,  however,  shows  in  his  fig.  2  that  many 
known  codas  give  results  closer  to  the  upper  bound  than 
tho  lower  bound.  This  is  to  be  expected  as  the  lower  bound 
represents  the  average  performance  of  all  possible  arbitrarily 
selected  codes  of  length  1,  including  the  worst  ones.  The 
difficult  problem  is  apparently  to  derive  an  algorithm  for 
finding  at  least  one  good  code  and  at  best  for  identifying 
the  optimum  code  for  any  given  length  1.  Slepian  (68) 
addresses  exactly  that  problem  and,  in  his  concluding  re¬ 
marks,  he  points  out  that  his  paper  may  have  “  raised  more 
questions  than  [it  has]  answered  ”.  He  further  concludes 
that  “  there  is  a  great  abundance  of  groups  of  arbitrarily 
large  order  than  can  be  examined  from  t  he  point  of  generating 
group  codes)  ”  and  ho  develops  a  number  of  theorems  that 
will  be  of  groat  help  to  research  workers  in  solving  the  im¬ 
portant  problem  of  finding  optimum  nonbinary  codebooks. 

A  closely  related  contribution  to  the  graphical  presenta¬ 
tions  of  the  mathematical  models  of  systems  with  nonbinary 
codebooks  comes  from  WJ  (65).  We  already  used  their 
exponential  bound  parameter  R0  in  equation  89  for  the  dis¬ 
cussion  of  binary  codebooks.  For  the  treatment  of  nonbinary 
codcbooks,  WJ  introduce  on  page  311,  equation  5.44b,  a 
nonbinary  exponential  bound  parameter  that  can  be  related 
to  the  units  of  this  paper. 


The  results  of  such  effort  are  given  in  fig.  27  for  one  fixed 
bit  error  ratio  (5b  =  10-3).  It  can  bo  seen  that  the  utility 
curves  gained  from  WJ’s  charts  supplement  well  the  curves 
gained  from  Shannon’s  reliability  function  and  from  Slepian’s 
charts.  A  numerical  comparison  shows  that  the  WJ  charts 
are  slightly  below  Shannon’s  upper  bound  but  well  in  line 
with  Slepian’s  bounds.  They  cover  a  much  wider  range 
of  bit  densities  and  code  lengths  than  either  of  the  other 
sources.  This  seems  to  bring  all  three  sources  in  very  good 
agreement.  Notice  that  WJ  derive  their  exponential  bound 
as  a  lower  bound  and  that  Slepian  proved  that  many  known 
codes  are  considerably  above  his  lower  bound.  It  may 
therefore  be  fairly  safe  to  use  the  WJ  curves  as  a  guideline 
for  the  utility  of  systems  with  nonbinary  codebooks  when 
operating  with  ideal  circuitry  over  a  Gaussian  channel. 

All  these  results  have  been  anticipated  by  theoretical 
investigations  and  some  of  these  results  are  contained  indivi¬ 
dually  in  any  one  of  the  many  contributions  of  other  authors. 
It  is  felt,  however,  that  the  utility  chart  offers  to  the  practical 
engineer  the  advantage  of  seeing  the  potential  tradeoff 
gains  and  costs  all  in  one  plot. 

Naturally,  the  engineer  will  also  be  interested  in  obtaining 
the  tradeoff  betweon  the  error  ratio  and  the  other  design 
parameters.  For  this  purpose  we  give  in  fig.  28  the  result 
of  using  the  WJ  charts  when  plotting  curves  for  codebooks 
with  constant  length,  but  operating  such  an  ideal  system 
with  different  transmission  quality.  For  facilitating  the 
comparison  with  earlier  (binary)  charts  we  chose  a  code 
length  of  40  dimensions  and  plotted  charts  for  seven  different 
bit  error  ratios  ranging  from  10-*  to  10_1°.  The  cost  in 
utility  for  decreasing  the  error  ratio  over  this  whole  range 
is  nearly  14  db  at  a  bit  density  of  —  15  db.  At  the  maximum 
of  the  curves  (around  zero  db  bit  density)  the  spread  is  only 
6  fib  and  it  decreases  further  at  the  right  side  of  the  chart.  A 
comparison  with  tho  curves  for  a  very  long  codebook  (1  = 
1000)  shows  that  large  codebooks  not  only  offer  the  advantage 
of  high  utility  but  they  also  assure  the  highest  quality  of 
information  transmission  without  any  significant  penalty 
in  utility.  The  two  curves  for  1  =  1000,  e«  =  1 0  ~ 1 ,  and 
1  =  1000,  5b  =  10-10  coincide  over  most  of  the  right  side 
of  the  utility  chart. 

Also  interesting  is  the  comparison  of  the  curves  of  fig.  28 
with  the  utility  lines  of  an  ideal  binary  system  (equation  34 
in  Part  II).  For  this  purpose  we  inserted  dashed  lines 


Figure  27. — Lower  bound  for  the  utility  of  a  system  with  a  fixed 
number  of  dimensions  ( after  Wozencraft  and  Jacobs). 
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Figure  28. — Utility  curves  of  ideal  multidimensional  systems  for 
various  error  ratios. 


for  the  utility  of  an  ideal  one-dimensional  binary  system 
for  the  same  error  ratios  as  the  ideal  multidimensional  non¬ 
binary  system.  One  can  see  that  for  high  error  ratios  (low 
transmission  quality)  the  complex  multidimensional  non¬ 
binary  system  does  not  offer  any  significant  advantage. 

After  having  discussed  the  most  general  mathematical 
models  of  multidimensional  nonbinary  systems,  it  is  important 
to  show  what  kind  of  codes  could  be  used  to  implement  such 
systems.  After  Rice  (50)  and  Shannon  (48,  49)  had  originally 
suggested  using  codes  with  randomly  selected  real  numbers 
as  code  elements,  Gilbert  (52)  was  possibly  the  first  to  investi¬ 
gate  some  simple  nonbinary  codebooks  with  integer  elements. 
He  explains  some  ternary  alphabets,  called  Slepian  alphabets, 
that  use  2  M  dimensions  (code  length)  and  words  with  M 
elements  having  letter  O  and  M  elements  having  either 
+  1  or  —1  as  letters.  A  typical  example  for  1  =  10  is  a  code 
word  ( +  1  0  1  — 10  -(-  100  0).  Gilbert  shows  that  such 

nonbinary  codebooks  can  be  very  efficient.  Systematic 
coding  procedures  for  digit  decoding  using  nonbinary  code 
elements  have  been  suggested  by  Golay  (49a),  and  a  few 
fundamentals  for  nonbinary  shift  register  codes  have  been 
included  by  Golomb  (55)  in  his  basic  report  on  digital 
sequences.  Loomba  (01)  published  an  interesting  comparison 
of  nonbinary  BC  codes  in  digit  decoding  versus  binary 
orthogonal  codes  in  message  decoding.  These  pioneering 
efforts  and  the  many  efforts  on  nonbinary  error-correcting 
codes  that  followed  them  are  not  of  direct  concern  in  this 
section.  All  the  models  that  have  been  discussed  above  are 
based  on  block  codes  with  MLD  decoding  procedure.  This 
requires  message  decoding  in  contrast  to  the  usual  symbol 
decoding.  Yet  certain  nonbinary  error-correcting  codes 
could  be  used  as  nonbinary  codobooks  in  our  sense.  The 
review  of  such  possibilities  is  beyond  the  scope  of  the  present 
paper. 

There  are,  however,  two  approaches  to  more  restrictive 
mathematical  models  that  have  been  successfully  brought 
into  the  form  of  numerical  charts.  The  first  approach 
deals  with  nonbinary  codebooks  that  use  multilevel  elements. 
It  is  due  to  Wozencraft  and  Jacobs  and  is  published  in  their 
book  in  the  form  of  two  charts  (figs.  5.17  and  5.18),  giving 


the  exponential  bound  parameter  as  a  function  of  the  energy 
ratio  per  dimension.  We  have  extended  WJ’s  fig.  5.17 
to  a  wider  range  of  E.v/N„,  called  x  in  our  fig.  29.  This 
variable  x  is  related  to  the  variables  of  the  utility  chart 
by  the  equations  shown  in  fig.  29.  The  exponential  bound 
parameter  R  as  a  function  of  x,  is  plotted  in  fig.  29  for 
various  values  of  q,  the  number  of  amplitude  levels  that  may 
be  used  for  the  elements  of  the  nonbinary  eodebook.  Evi¬ 
dently  the  curve  for  q->oo  is  identical  with  the  curve  used 
for  plotting  figs.  27  and  28. 

There  are  two  useful  approximations  to  the  curve  R0*  (x, 
q— >  co ) .  They  are  given  in  fig.  29  by  the  equations  2  and 
3  while  equation  1  is  the  analytical  form  of  the  correct 
curve  for  q-»-co.  With  the  help  of  this  extended  chart 
drawn  to  a  very  accurate  scale,  we  plotted  in  fig.  30  the  utility 
charts  of  quantized  nonbinary  codebooks,  used  as  multilevel 
codebooks.  The  utility  curves  of  fig.  30  are  plotted  for 
e/i  10- 3  and  1  =  40.  The  signal  base  is  =  0.5,  as  usual. 
The  curve  for  q  =  2  is  identical  with  the  curve  for  the  same 
parameters  plotted  in  fig.  22  for  binary  codebooks.  The 
same  exponential  bound  has  been  used  in  both  cases.  The 
curves  for  q  >  2  show  that  the  principal  advantage  of  multi¬ 
level  nonbinary  codebooks  is  in  an  extension  of  the  utility 
curves  to  larger  positive  bit  densities.  The  differences  in 
the  utility  of  the  various  curves  at  the  left  side  are  due  to 
some  geometrical  characteristics  of  quantized  real  codes 
in  the  1-dimensional  vector  space.  WJ  developed  an 
optimization  procedure  for  assigning  different  probabilities 
to  the  different  amplitude  levels  when  selecting  them  for 
elements  in  the  code  words.  With  the  help  of  this  procedure 
all  curves  may  be  brought  to  the  same  level  at  the  left 
side,  i.e.,  to  the  level  of  the  curve  for  q->ao.  Figure  5.18 
in  WJ  shows  the  R„*  resulting  in  that  case. 

Figure  30  shows  Shannon’s  upper  bound  for  this  case, 
gained  from  Slepian’s  curves  by  interpolation.  Remember 
that  the  WJ  model  is  a  lower  bound.  To  show  the  wide 
margin  that  still  prevails  between  the  lower  bound  of  the 
theoretical  models  and  the  practical  results  of  one  of  the 
first  experimental  systems  that  have  been  tested,  we  also 
inserted  the  operating  line  for  the  Lincoln  Laboratory’s 
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Figure  2!). — Plot  of  the  reliability  of  multidimensional  systems  with 
quantized  elem  cuts. 
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self-regulating  error  correction  coder-decoder  (SECO)  system 
that  operates  with  q  =  32  levels  (section  F7,  Part  II). 
It  had  been  classified  as  a  one-dimensional  system  because 
it  uses  symbol  decoding  and  not  message  decoding.  For 
its  class  the  SECO  is  efficient  equipment,  but  the  combined 
inefficiencies  of  the  digit  decoding  method,  the  waveform 
spectra,  the  synchronization  methods,  and  many  other 
engineering  problems  place  it  in  utility  about  10  db  below 
the  mathematical  model.  Interesting,  however,  is  the  fact 
that  a  system  of  this  complexity  has  been  built  at  all  and 
that  in  its  first  attempt  it  conies  that  close  to  t  he  theoretical 
optimum.  Notice  also  that  the  slope  of  the  operating  lino 
of  the  SECO  system  is  smaller  than  the  slope  of  the  mathe¬ 
matical  model  (the  utility  chart  marked  q  =  32).  This 
makes  the  practical  system  more  efficient  at  higher  bit 
densities  than  at  lower  ones.  There  are  many  possible 
reasons  for  this  behaviour  ;  some  of  them  could  be  in  the 
kind  of  measurement  methods  applied  for  the  tests  of  SECO. 

The  second  approach  to  arrive  at  numerical  charts  for 
more  restrictive  mathematical  models  of  systems  with 
nonbinary  codebooks  is  due  to  Peterson  and  Ivasami  (65). 
These  authors  compared  the  following  classes  of  nonbinary 
codebooks : 

A.  S-Classes  :  in  lino  with  Shannon’s  (59)  definitions. 

B.  D -Classes  :  generally  called  polyphase  codes  and  also 

investigated  by  the  following  authors : 
Viterbi  (65),  Stiffler  and  Viterbi  (65),  and 
Reed  and  Scholtz  (65,  66). 

C.  C-Classes  :  These  are  double  encoded  systems  where 

the  )3-eloments  of  the  first  codebook  (fig.  16) 
are  further  encoded  into  a  waveform 
library  (not  necessarily  of  a  finite  alphabet) 
where  each  waveform  consists  of  an  M-tuplo 
of  real  number  components  such  that  all 
waveforms  have  the  same  average  power, 
and  thus  are  represented  by  vectors  with 
end  points  on  an  M-dimensional  sphere. 

Peterson  and  Ivasami  (65)  use  exactly  the  same  reliability 
charts  as  Shannon  (59)  and  as  we  discussed  in  fig.  23.  Their 
results  can  therefore  directly  be  converted  to  utility  charts 
following  the  previously  described  procedure.  The  results 
indicate  that,  for  very  small  SNR  (power  contrast  o-),  all 
the  classes  above  merge  into  the  same  curves  for  lower 
and  upper  bound  or  for  the  combined  bound.  These  bounds 
are  the  ones  that  we  derived  as  approximation  for  a  1 
from  Shannon’s  (59)  paper  and  that  we  reproduced  in  Fig.  25. 
For  a  values  closer  to  unity  and  above  unity,  we  find  that 
again  all  curves  merge  for  the  same  bound  into  the  same  curve 
or  point  when  the  bit  density  goes  to  zero,  i.e.,  at  the  left 
side  of  the  curve. 

The  principal  results  of  Peterson  and  Ivasami  (65)  are 
therefore  the  differences  in  the  reliability  E  plotted  in  the 
charts  for  A  =  1  (a  =  0  db),  A  =  2 [a  =  +6.02  db),  A  = 
4( a  =  +12.04  db),  A  =  8(a  =  + 18.06  db),  and  A  =  16(cr  = 
+  24.08  db)  for  values  of  the  rate  parameter  larger  than 
0.1  (D  >  —5.4  db).  This  area  corresponds  to  the  centre 
and  the  right  side  of  the  utility  chart,  again  an  indication 
that  nonbinary  codebooks  are  of  importance  for  systems 
with  high  bit  density.  In  that  area  we  recognize  that  S-classes 
(i.e.,  the  real  linear  codes),  have  the  highest  reliability  (and, 
correspondingly,  the  highest  utility  for  any  given  signal 
contrast  a)  of  all  the  classes,  no  matter  whether  one  takes 
the  lower  bound  or  the  upper  bound.  The  C-classes  (i.e.,  the 
double  encoded  systems)  are  significantly  better  than  the 
D -classes  and  in  both  classes  the  size  of  the  waveform 
alphabet  (q)  becomes  important  at  the  higher  bit  densities. 
Naturally,  the  tendency  is  toward  increasing  utility  for 
increasing  q  value.  Beyond  these  general  observations 
the  reader  should  have  no  difficulties  in  finding,  for  any 
particular  selection  of  a  class  of  codes  and  for  any  specific 
sot  of  values  for  e«  ,  1,  q,  and  D,  the  utility  u  by  using  equa- 
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tion  76  (Peterson  and  Ivasami  use  the  same  R  as  RSH  in 
this  paper)  and  equation  77  and  the  corresponding  curve 
(for  specific  q  value  and  for  the  code  class)  in  Peterson. 

There  arc  many  other  important  investigations  of  the 
mathematical  background  of  nonbinary  codebooks.  The 
previously  mentioned  paper  by  Gallager  (65)  gives  a  most 
complete  coverage  of  the  whole  field.  Much  interest  resulted 
from  the  so-called  n-orthogonal  codes  that  wrere  investigated 
by  Viterbi  (65)  as  polyphase  codes,  by  Stiffler  and  Viterbi  (65) 
as  q-orthogonal  signals,  and  by  Reed  and  Scholtz  (65,  66) 
as  n-orthogonal  phase  modulated  codes.  Multiphase  codes 
for  radar  purposes  wyere  proposed  earlier  by  Heimiller  (61) 
and  Frank  and  Zadoff  (62).  N-orthogonal  codes  are  closely 
related  to  the  simplex  codes  (to  bo  discussed  in  the  next 
section)  as  pointed  out  by  Weber  (66)  and  I.  Jacobs  (67). 
The  latter  publication  offers  a  very  interesting  comparison 
of  n-orthogonal  phase  modulation  with  many  other  non- 
binarv  systems  and  concludes  on  page  83  “  that  n-orthogonal 
phase  modulation  is  generally  inferior  to  more  conventional 
techniques  ”. 

Interesting  progress  towards  specific  nonbinary  codebooks 
may  also  come  from  a  further  expansion  of  the  theory  of 
binary  sequences  to  sequences  with  nonbinary  elements. 
Kurz  (61)  postulated  an  efficient  code,  the  Minimax  code, 
that  may  form  the  basis  for  future  developments  of  nonbinary 
codebooks  combined  with  orthogonal  waveform  libraries, 
possibly  along  the  lines  of  Peterson  and  Ivasami  (65)  class  C 
of  codes.  Hsieh  atid  Hsiao  (64)  show'  a  method  of  construc¬ 
ting  n-ary  codes,  particularly  ternary  codes  from  binary 
sequences  using  orthogonal  functions.  Much  hope  may 
also  be  placed  on  the  generalized  Barker  Sequences  which 
received  a  thorough  fundamental  treatment  by  Golomb 
and  Scholtz  (65).  Barker  sequences  were  originally  suggested 
for  group  synchronization  of  binary  digital  systems  (Barker 
53).  Delong,  Jr.  (59)  extended  their  theory  to  three-phase 
codes.  The  generalization  by  Golomb  et  al  introduces 
complex  numbers  having  an  absolute  value  of  “  1  ”.  This 
makes  them  very  suitable  for  codebooks  of  the  D -class  of 
Peterson  and  Kasami  (65).  Of  additional  interest  may  be  a 
note  by  S.  H.  Chang  (66b)  on  nonbinary  orthogonal  codes 
and  the  work  by  Levitt  and  Wolf  (65)  on  the  correlation 
properties  of  multilevel  cyclic  sequences. 

The  theoretical  superiority  of  codes  that  permit  any  real 
number,  not  just  integers,  to  function  as  code  elements  jit 
has  been  established.  Our  fig.  30  shows  the  advantage  to  be 
gained  by  going  with  q  toward  infinity.  The  practical 
engineering  possibilities  to  implement  such  codebooks  are 
under  investigation  by  W.  H.  Pierce  (68)  for  the  linear-real 
codes  (originally  anticipated  by  Kelly  (60)  and  then  devised 
by  W.  H.  Pierce  (66).  Schneider  (68)  investigated  the  en¬ 
gineering  possibilities  for  Slepian’s  (65a)  permutation 
modulation.  Ziv  (63)  investigated  the  coding  and  decoding 
for  time-discrete  amplitude-continuous  memoryless  channels 
in  a  doctoral  thesis  and  his  results  should  be  of  interest  to 
developers  of  linear-real  coding  equipment. 

Concluding,  we  may  predict  that  the  multidimensional 
information  transmission  systems  with  nonbinary  codebooks 
will  become  of  growing  importance  in  the  future.  Their 
mathematical  models  are  wrell  developed.  The  code  genera¬ 
tors  will  have  to  operate  on  a  systematic  basis  to  avoid 
excessively  large  memory  requirements  for  codebooks. 
The  codes  will  require  code  lengths  of  one  hundred  and  more 
elements  and  the  elements  or  samples  will  have  to  be  selected 
from  lai’ge  alphabets. 

Iv8.  Mathematical  Models  of  Systems  with  Simplex  Codebooks 

In  subsection  J2  we  gave  a  capsule  history  of  the  sphere¬ 
packing  problem  and  mentioned  that  regular  polytopes 
(Coxeter  48,  62)  were  investigated  as  the  geometrical  figures 
in  an  n-dimensional  vector  space  that  would  lead  to  “  good  ” 
codebooks  in  the  sense  of  having  the  maximum  average 
distance  between  all  code  words.  First  Gilbert  (52),  and 
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later  Basore  (59)  and  Stutt  (59,  60),  developed  regular 
polytope  codes.  The  term  “  polytope  ”  should  be  preferred 
to  the  term  “  polyhedron  ”,  which  is  reserved  for  a  polytope 
in  three-dimensional  space.  The  term  “  simplex  ”  designates 
a  subclass  of  n-dimensional  polytopes  with  elements  that  are 
simplexes  of  n-1  dimensions.  A  regular  simplex  is  basically  a 
simplex  with  all  “  edges  ”  being  equal.  The  exact  definitions 
of  these  terms  in  n-dimensional  space  are  moro  complex 
and  should  be  checked  in  the  book  of  Coxeter  (62). 

Stutt  (59)  gave  a  procedure  for  developing  an  “  equilateral 
polyhedron  ”  (regular  simplex)  in  a  minimum  number  of 
dimensions  and  computed  a  number  of  charts  that  can  be 
easily  converted  to  utility  charts.  Regular  simplex  codes 
have  utility  curves  which  are  only  slightly  above  the  utility 
curves  for  orthogonal  codes  (figs.  20  and  21).  Their  code¬ 
books  are  nearly  square  with  l  +  1  words  for  l  dimensions. 
This  gives  them  a  slightly  larger  bit  density  for  any  given  l 
than  orthogonal  codes  can  offer.  Simplex  codes  have  a 
slightly  negative  cross-correlation  matrix  and  therefore 
give  a  slightly  higher  utility  than  orthogonal  codes  when 
operated  with  an  MLD  processor. 

Balakrishnan  (60)  showed  that  (a)  when  one  out  of  a 
finite  number  of  signals  (represented  by  vectors  in  /-dimen¬ 
sional  space)  is  transmitted  over  a  Gaussian  channel,  the 
optimum  /-dimensional  complex  for  placing  the  signal 
vectors  is  the  regular  simplex,  and  that  (b)  all  signal  vectors 
should  end  at  the  corners  of  the  regular  simplex.  Balak¬ 
rishnan  (61)  investigated  the  influence  of  the  SNR  (defined 
in  geometrical  terms)  on  the  optimal  choice  of  a  codebook 
and  proved  that  the  regular  simplex  is  a  local  maximum 
independent  of  the  SNR.  Balakrishnan  and  Taber  (62) 
gave  error  probability  curves  and  curves  for  the  communica¬ 
tions  efficiency  of  simplex  systems  and  compared  the  results 
for  simplex  systems  with  those  for  biorthogonal  systems, 
showing  again  that  the  regular  simplex  coding  yielded  the 
lowest  possible  error  ratio  for  a  given  SNR  (defined  as  signal 
power  over  noise  power  density).  Balakrishnan  (65)  related 
his  earlier  results  to  the  problem  of  signal  selection  for  space 
communications  channels  and  stressed  again  the  need  for 
proving  that  the  regular  simplex  is  an  “  absolute  or  global  ” 
maximum  independent  of  SNR,  or  for  disproving  this 
conjecture. 

Landau  and  Slepian  (66)  proved  finally  that  the  regular 
simplex  code  is  indeed  the  optimum  code  for  all  SNR  when 
the  eodobook  matrix  has  /  dimensions  and  M  =  /  +  1  code 
words.  They  also  showed  that  the  biorthogonal  code 
likewise  is  optimum  when  the  code  book  has  six  words 
and  three  dimensions  (M  =  21)  and  that  a  code  consisting 
of  the  midpoints  of  the  faces  of  the  regular  dodecahedron 
i  is  also  optimum  for  M  =  12  and  /  =  3  (M  =  41).  “  Optimum  ” 

in  all  these  derivations  means  that  the  application  of  these 
codes  over  a  Gaussian  channel  in  any  system  with  an  MLD 
receiver  and  otherwise  ideal  circuitry  will  yield  the  lowest 
possible  error  probability  for  any  given  SNR. 

L.  Experimental  Multidimensional  Systems 

Compared  with  the  strong  development  effort  that  takes 
place  in  the  area  of  one-dimensional  nonbinary  systems, 
there  is  not  yet,  much  activity  in  experimental  multidimen¬ 
sional  systems. 

LI.  DIOI LOCK  Data  Transmission  System 

The  development  of  a  system  using  binary  orthogonal 
sequences  as  nonbinary  signals  was  widely  publicized. 
Called  the  DIGILOCK  system,  it  is  based  on  a  mathematical 
model  published  by  Viterbi  (61),  which  was  used  for  the 
utility  charts  of  fig.  2,  Part  I.  The  development  work  on 
DIGILOCK  was  originally  sponsored  by  the  Jet  Propulsion 
Laboratory,  and  the  fundamentals  of  the  system  were 
published  by  Sanders  (59a  ;  60b).  The  experimental 
version  of  the  system  was  designed  to  operate  with  orthogonal 
sets  of  a  modified  Reed-Miiller  code  type  (first  order)  or  with 
the  same  set  in  biorthogonal  version.  An  MLD  receiver 
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was  used.  Viterbi  (61)  showed  how  codes  derived  from  PN 
sequences  can  be  used  as  orthogonal  sets.  Sanders  (65) 
gave  another  review  of  the  DIGILOCK  fundamentals 
and  discussed  the  synchronization  problem  and  its  inter¬ 
connection  with  the  choice  of  the  waveform  alphabet  (binary, 
in  this  case).  From  all  these  reports  we  learn  that  biphase 
modulation  is  used  on  the  radio  carrier  in  the  experimental 
version.  It  is  reported  that  the  laboratory  test  results 
placed  the  utility  2  db  below  the  theoretical  value  (fig.  2, 
Part  I).  No  information  Ls  supplied  about  the  bandwidth 
occupancy  ;  thus  it  is  impossible  to  define  the  bit  density 
exactly.  The  values  used  in  fig.  2  seem  to  be  a  fair  assump¬ 
tion  for  biphase  modulation.  It  is  not  reported  that  the 
experimental  version  of  DIGILOCK  had  been  tried  in  field 
tests  with  synchronization  over  a  noisy  channel.  It  is, 
however,  reported  that  a  special  version  of  the  DIGILOCK 
telemetry  system  passed  its  flight  test  on  December  4, 
1961  (Collins  61).  A  more  complete  description  of  this 
operational  system  (Jaffe  62)  revealed  that  the  system 
was  usod  on  the  Blue  Scout,  Jr.  USAF  rocket  in  parallel 
with  a  standard  FM/FM  telemetry  link.  A  16-dimensional 
modified  Reed-Miiller  code  was  used  in  the  biorthogonal 
mode.  Phase  modulation  of  ±60°  was  used  on  the  radio 
frequency  carrier  to  keep  sufficient  carrier  power  at  the  carrier 
frequency  itself,  so  that  a  phase  lock  loop  could  operate 
safely  at  the  receiver. 

During  the  field  test  the  system  operated  with  a  total 
average  transmitter  power  of  250  mW.  The  maximum 
range  is  100,000  miles  ;  the  transmitter  antenna  gain,  0  db  ; 
and  the  receiving  antenna  aperture  area,  850  ft,2  =  79  m2. 
This  gives  a  signal  power  at  the  receiver  input  (Filipowsky 
and  Muehldorf  65a)  of 


Pr  A,  0.25  (79) 
4ttH-  477(2.59)  (10'“) 


5.96  X  10~17  W 


(90) 


The  noise  power  at  the  receiver  input  may  be  calculated 
from  the  receiver  bandwidth  B  and  from  the  receiver  noise 
figure  NF  increased  (after  conversion)  by  antenna  and 
coupling  device  noise  temperature,  using  the  formulas  : 

PA-  =  KTjv  B  ;  T,v  =  290(NF  -  1)  +  Tc  .  (91) 

with  K  =  1.38(10" 22)  Watt  second  per  degree  Kelvin. 

Jaffe  gives  a  noise  figure  of  2.82  and  a  bandwidth  of  3200 
Hz.  We  may  assume  a  noise  temperature  of  about  70° 
for  the  external  noise  contributions,  making  the  operational 
noise  temperature  about  600  °K. 

Pa-  =  1.38(10-23)  (600)  (3200)  =  2.65(10-»7) 

This  allows  a  power  contrast  of  5.96/2.65  =  2.25  or  a  = 
+  3.52  db.  Before  we  can  calculate  the  utility,  wo  need  the 
bit  density.  The  system  operates  with  67  information 
bits  per  second,  but  every  21st  word  is  a  frame  synchroniza¬ 
tion  word  making  the  actual  message  rate  R,„  =  67(20/21)  = 
63.8  bps.  The  bit  density  is  therefore 

R,„  63.8 

D=  — = - =  0.0199  or  D-“  =  -17.01  db 

B  3200 


Notice  that  the  system  uses  a  codebook  with  1  =  16 
dimensions  and  M  =  32  code  words.  The  signal  base 
per  dimension  is  rather  large  : 


1  1R  16(67) 


=  14.9 


Notice  that,  for  the  calculat  ion  of  the  signal  base,  wo  must 
take  the  bit  rate  of  67  bps  (also  called  the  throughput)  ; 
whereas,  for  calculating  the  bit  density  D,  we  can  only  count 
the  transmitted  message  bits,  deducting  any  bits  for  frame 
synchronization ;  t  hese  bits  contribute  to  the  total  syn¬ 
chronization  and  acquisition  performance.  If  those  extra 
bits  are  for  some  “  housekeeping  ”  purposes  and  can  bo 
freely  used  for  messages  without  disturbing  the  performance 
of  the  system,  the  message  rate  should  be  equal  to  the 
throughput. 
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Knowing  D  and  a  for  the  operation  of  the  system  at  the 
maximum  range,  we  can  now  find  the  utility  as 

u"  =  D**  —  a<,b  =  —17.01  —  3.52  =  -20.52  db 

The  system  uses  roughly  25  per  cent  of  the  transmitter 
power  for  the  carrier  channel  (phase  lock  loop  bandwidth  = 
20  Hz).  This  requires  that  1.9  db  ho  deducted  from  the 
operational  utility.  The  operating  point  of  tlio  system, 
when  operating  with  full  power,  is  therefore  at  u  =  —20.7  db 
and  D  =  —17  db.  Because  the  designers  need  a  design 
margin,  they  specify  6  db.  Thus  the  utility  should  not  go 
below  u  =  —14.7  db.  At  this  place  the  system  still  has 
an  operational  margin  of  about  6  db  against  the  design  goal 
of  u  =  —9  db,  where  the  theoretical  error  ratio  should  be 
10“ 0  or  bettor.  That  design  goal  was  specified  as  f)s  a  =  8, 
using  Sander’s  communications  efficiency  (Sanders  59d,  60a) 
which  is  (with  reservations)  the  reciprocal  value  of  the  utility. 

The  actual  receiving  technique  used  a  predetection 
recording  method  and  a  signal  processing  maximum  likelihood 
decoding  procedure  with  the  help  of  a  commercial  computer. 
By  these  rather  unconventional  methods  it  was  possible  to 
make  full  use  of  the  large  signal  base  (by  integrate  and  dump 
procedures)  and  to  perform  truly  an  MLD  decoding  operation. 
It  was  interesting  for  the  designers  to  test  the  receiver 
portion  under  worst  ease  conditions  by  reducing  the  SNR 
for  a  laboratory  test.  They  adjusted  the  SNR  in  a  3200  Hz 
band  after  a  crystal  filtei  to  the  low  value  of  —12.0  db, 
corresponding  to  —13.2  d  for  the  message  channel  above. 
They  achieved  an  error  rat  •.  of  0.02  per  word  or  0.01  per  bit 
(now  counting  the  frame  synchronization  bits  as  message 
bits).  The  decoding  was  performed  by  the  computer. 
This  worst  case  situation  yields  a  utility  of  u  =  —16.8  + 
13.2  =  —3.6  db.  This  is  only  about  2.2  db  below  the 
theoretical  value  of  u  =  —1.4  db,  which  coidd  be  achieved 
with  D  =  —16.8  db  and  ea  =  10-2. 


L2.  Block  Coded  Telemetry  Systems  of  Jet  Propulsion 
Laboratory 

The  staff  of  the  Jet  Propulsion  Laboratory  has  consistently 
improved  their  experience  with  multidimensional  nonbinary 
systems.  Much  of  this  experience  resulted  from  original 
work  on  ranging  codes  (59,  60,  65).  A  system  block  diagram 
and  some  logical  design  information  about  a  biorthogonal 
telemetry  system  of  16-dimensions  for  45  bps  are  supplied 
in  another  report  (61a).  To  improve  the  demodulator  of 
the  MARINER  2  data  system,  JPL  made  an  investigation 
to  find  practical  design  approaches  to  matched  filter  correla¬ 
tors  (Futz  63).  Futz’s  (63)  note  concludes  that  the  usual 
mechanization  of  a  coherent  demodulator  is  not  equivalent 
to  the  commonly  used  mathematical  model.  The  investi¬ 
gation  showed  that  it  is  possible,  by  correctly  matching  a 
filter  to  the  signal  waveform,  to  improvo  the  performance 
of  the  Mariner  2  demodulation  by  0.9  db.  The  practical 
realization  of  the  matched  filters,  however,  presented  diffi¬ 
culties. 

An  interesting  article  (63b)  describes  a  large  number  of 
trial  runs  of  a  biorthogonal  telemetry  system  (published 
earlier  in  JPL  publications).  The  results  of  these  trial 
rims  performed  in  the  laboratory  with  a  noise  generator 
showed  that  the  SNR  ratio  to  obtain  a  particular  error 
probability  was  within  0.3  db  of  that  predicted  theoretically. 
The  system  subjected  to  these  trial  runs  was  the  45  bps 
telemetry  system  mentioned  above.  It  was  exactly  timed 
to  a  word  period  of  110  ms,  to  a  bit  period  22  ms  (n  =  5), 
and  a  chip  period  of  6.875  ms  (1  =  16).  The  chips  biphase- 
modulated  a  4  KHz  rectangular  carrier.  A  receiver  band- 
filter  restricted  the  transmission  bandwidth  to  5.4  KHz 
(1.3  to  6.7  KHz).  This  gives  a  bit  density  of 


1 

TnB 


105 

22(5.4)  (I03) 


=  0.00841  (or  -20.74  db) 


The  utility  (or  its  reciprocal  value  E/N„)  is  shown  as  a 
function  of  the  bit  error  probability,  and  the  experimental 


values  agree  within  a  fraction  of  a  decibel  with  the  theoretical 
values.  The  results  of  these  laboratory  tests  are  gratifying 
since  they  confirm  the  correctness  of  the  mathematical  model. 
Yet,  the  same  problem  arises  as  in  the  case  of  the  tests  of 
the  DIGILOCK  system  reported  by  Jaffe  (62).  The  bit 
density  in  the  tests  actually  used  is  —20.74  db.  The  reason¬ 
able  minimum  bit  density  for  a  biorthogonal  system  of 
n  =  5  can  bo  taken  from  fig.  2  as  about  —4  db.  (The 
theoretical  minimum  bit  density  for  n  =  4  in  an  orthogonal 
system  would  be  —3  db  in  a  low-pass  channel  with  sin  x/x 
waveforms,  corresponding  to  j3c  =  0.5).  Because  of  the 
phase  modulation  the  designers  preferred  the  much  larger 
f} c  —  n/lD  =  37.2  (or  15.7  db),  but,  since  they  use  it  only 
as  a  correlation  improvement  (like  in  idoal  binary  systems), 
no  improvement  in  utility  results  from  this  large  bandspread¬ 
ing  effort.  The  note  gives  no  information  about  field  tests 
or  tests  on  space  (lights  but  it  discloses  details  on  the  develop¬ 
ment  of  the  digital  circuit  modules.  It  may  be  of  interest 
that  the  decoder,  similarly  as  in  the  USAF  DIGILOCK 
system,  is  a  digital  signal  processor  which  resamples  each 
received  chip  55  times  and  quantizes  each  sample  into  libit 
words  (2048  levels).  This  comes  very  close  to  an  idoal 
MLD  receiver.  It  is  therefore  no  suprise  that  the  laboratory 
tests  are  indeed  within  0.3  db  of  the  theoretical  model. 

Springett  (65)  stressed  the  importance  of  the  JPL  research 
in  multidimensional  nonbinary  systems  and  in  related 
areas  for  the  further  development  of  practical  space  telemetry 
and  command  systems.  It  is  interesting  to  note  that  he 
uses  a  normalized  SNR  defined  as  the  signal  power  over  the 
noise  power  density  S /N  „  ,  a  magnitude  defined  in  equation  1 2 
in  Part  I  as  the  characteristic  rate  of  a  system  and  designated 
by  the  symbol  “  p  ”.  The  importance  of  the  parameter 
has  been  stressed  independently  of  Springett  by  Filipowsky 
and  Muehldorf  (65a),  where  it  was  called  signal  contrast 
frequency.  Contrary  to  the  truly  normalized  SNR  terms,  the 
power  contrast  (a)  and  the  energy  contrast  (e  =  E/N<,),  this 
so-called  “  normalized  SNR  ”  is  actually  a  physical  magnitude 
with  the  dimension  of  a  frequency  or  a  rate,  i.e.,  with  the 
dimension  (second)-1.  We  therefore  suggest  calling  it 
“  characteristic  rate  ”,  not  “  normalized  SNR  ”.  This  term 
would  also  be  significant  since  it  can  be  easily  shown  (in  addi¬ 
tion  to  the  relations  in  equation  12)  that  this  magnitude  p  is 
related  to  the  utility  in  the  following  form  : 

D  R  N  R 


Springett  (65)  shows  that  the  state-of-the-art  at  that  time 
permitted  communications  from  MARS  to  Earth  at  p  = 
30.5  db-Hz,  or  at  a  rate  of  1121  bps,  with  a  hypothetical 
system  that  could  operate  with  a  utility  of  unity  (0  db 
or  100-percent  efficiency).  The  numerous  references  in 
Springett  (65)  will  bo  helpful  to  readers  interested  in  the 
engineering  evolution  of  space  data  transmission  systems. 

Saliga  (67)  reports  an  advanced  space  telemetry  system 
with  biorthogonal  codes  of  127  dimensions  carrying  8-bit 
words.  The  experimental  system  was  developed  along 
the  lines  of  DIGILOCK  and  of  the  JPL  telemetry  systems 
for  5-bit  words  that  we  discussed  above.  The  significant 
improvement  is  due  to  a  now  synchronization  and  timing 
subsystem,  the  Carrier  Coherent  Telemetry  Bate  (CCTIl) 
subsystem.  The  improved  synchronization,  acquisition,  and 
timing  methods  aro  based  on  the  experience  with  ranging 
codes  (Titsworth  64).  These  methods  are  explained  in 
detail  in  Saliga’s  1967  paper.  The  result  of  these  improve¬ 
ments  is  an  acquisition  threshold  at  u  =  —0.5  db  and  a 
worst  case  acquisition  time  of  22.5  seconds  for  a  data  rate 
of  174  bps.  At  less  utility  (higher  energy  contrast)  the 
acquisition  time  can  be  reduced  to  7.5  seconds,  corresponding 
to  5  telemetry  frames.  The  synchronization  code  consumes 
less  than  6.4  percent  of  the  available  power. 

The  effective  bandwidth  for  SNR  measurement  is  specified 
by  Saliga  as  twice  the  “  symbol  rate  ”.  This  amounts  to 
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about  11  KHz  at  a  throughput  of  174  bps,  considering  that 
each  chip  has  two  symbols  (for  clocking  purposes)  and  each 
word  has  127  chips.  At  this  bandwidth  a  power  contrast  of 
o  =  — 14.5  db  is  required  for  a  word  error  ratio  of  0.1  (bit 
error  ratio  >f  5. 1 0 _  -) .  This  gives  the  system  a  maximum 
bit  density  (throughput)  of  127/11000  -  0.01156  (  —  19.4  db) 
and  a  maximum  utility  of  u  =  —19.4  -f-  14.5  —  —4.9  db. 
The  signal  base  per  dimension  is  smaller  than  for  the  5-bit 
system  (/Jc  =  6.45)  but  still  large  when  compared  with  the 
theoretical  minimum  of  0.5. 


L3.  Experimental  Development  in  Other  Institutions 

Though  the  most  significant  early  development  efforts  in 
I  multidimensional  nonbinary  systems  originated  in  the  NASA 
laboratories  j  (primarily  .TPL),  many  other  institutions  and 
I  industrial  laboratories  are  now  participating  in  the  explora¬ 
tion  of  such  systems.  The  following  short  chronological 
I  list  may  give  a  superficial  survey  of  the  magnitude  of  this 
effort.  A  reliable  comparison  of  the  results  is  impossible, 
due  to  the  scarce  data  published  so  far. 

Price  and  Greene  (58)  published  the  first  comprehensive 
description  of  a  binary  communications  system  that  used 
| two  orthogonal  (or  nearly  orthogonal)  shift  register  sequences 
as  waveforms  with  large  signal  base.  The  system  received 
|the  code  name  RAKE.  Though  the  system  is  essentially 
'a  binary  system,  the  waveforms  themselves  form  a  multi¬ 
dimensional  codebook  with  two  code  words  and  l  =  2*  —  I 
dimensions  with  k  being  an  integer.  These  characteristics, 
together  with  the  matched  filter  extractor  and  maximum 
likelihood  decider,  make  the  RAKE  system  clearly  a  pre¬ 
cursor  of  the  systems  with  larger  codebooks  to  bo  discussed 
here.  The  5B  references  in  the  Price  and  Green  (58)  paper 
give  a  good  I  cross  section  of  the  literat  ure  prior  to  1958. 
The  autocorrelation  properties  of  the  PR  sequences  (Camp¬ 
bell  59)  made  the  RAKE  system  particularly  suitable  for 
transmission  over  multipath  channels. 

J  Hove  and  Lightfoot  (63)  contributed  a  paper  on  the  design 
of  matched  filters  for  high  chip  rates  using  act  ive  RC  cir¬ 
cuitry.  The  1 7  references  in  t  hat  paper  may  serve  as  a  review 
pf  the  precursor  technology  of  LSI  matched  filter  extractors. 
A  Russian  paper  (Mitryayev  63)  shows  the  interest  of  engi- 

! leers  in  that  1  country  in  systems  with  group  codes  as  code- 
>ooks.  Ballard  (63)  developed  an  experimental  system 
ailed  ORTHOMUX,  which  uses  sets  of  orthogonal  binary 
equences  as  carriers  in  a  mult  iplex  system.  Notice  that  the 
resulting  composite  signal  forms  transmission  words  of 
nonbinary  elements.  Lachs  (63)  reported  a  simulation  of 
multidimensional  nonbinary  codebooks  by  a  digital  computer. 
His  investigation  aimed  at  the  maximization  of  the  minimum 
distance  between  code  words  in  a  multidimensional  vector 
space. 

In  1964  the  interest  in  applying  binary  PN  sequences  in 
communications  systems  continued  with  a  paper  (Shepertycki 
64)  on  their  I  application  as  a  random  message  source  in 
tjelemetry  error  measurements.  Blasbalg  et  al  (64)  presented 
the  concept  pf  a  random  access  communications  system 
that  applied  large  sets  of  PN  sequences  in  combination  with  a 
frequency  offset  of  subcarrier  frequencies  as  addresses  for 
several  thousand  receivers  that  might  subscribe  to  the 
participation  in  such  a  system.  Computer  simulation  was 
used  to  explore  the  system  performance  with  2,  5,  10,  and 
15  equal  power  simultaneous  talkers.  Lightfoot  and  Kogut 
(if>4)  and  Lightfoot  (65)  described  the  development  of  an 
experimental  data  transmission  terfninal  for  HF  radio 
frequency  transmission  which  makes  use  of  PR  sequences  in  a 
similar  mode  to  the  RAKE  system.  The  novel  feature  is  a 
programmable  matched  filter  consisting  of  100  separate  active 
RC  circuits.  The  particular  detection  technique  applied 
in  this  experimental  terminal  minimizes  the  synchronization 
requirements  between  transmitter  and  receiver. 

A  detailed  description  of  an  experimental  terminal  for  a 
quantized  pulse  position  modulation  (QPPM)  system  is 
given  by  Huber  (65).  QPPM,  also  called  MTSK  (mul¬ 
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tiple  time  shift  keying),  or  as  in  this  case,  monocycle  posi¬ 
tion  modulation,  has  become  the  classical  example  for 
describing  a  typical  orthogonal  nonbinary  communica¬ 
tions  system  (section  K5  above).  The  report  shows  that 
systems  of  this  kind  have  attractive  characteristics  for 
tactical  army  applications.  Wang  (65)  found  a  connna- 
free  square  binary  codebook  for  a  telemetry  system  of  the 
kind  described  at  the  beginning  of  this  section.  This 
method,  which  is  based  on  results  originally  published  by 
Stiffler  (62),  is  applied  to  a  biorthogonal  codebook  of 
1  =16,n  =  5.  Laboratory  tests  show  good  agreement  with 
the  theory  and  prove  that  systems  with  such  codebooks 
have  improved  synchronization  characteristics.  Greim  et 
al  (65)  are  the  authors  of  a  large  test  report  which  was  men¬ 
tioned  earlier.  It  compares  the  performance  of  two  novel 
experimental  systems  which  incorporate  certain  nonbinary 
features  (DEFT  and  KATHRYN)  with  commercially 
available  systems.  All  systems  are  specifically  designed 
for  operation  over  long  distance  radio  links  in  the  HF  bands. 
The  report  does  not  describe  monosignal  multidimensional 
nonbinary  systems,  in  which  we  are  interested  here,  but  may 
be  useful  in  systems  comparisons. 

An  interesting  data  transmission  system  for  geophysical 
research  is  the  pulse  morse  code  modulation  (PMCM)  system 
described  by  Toney  and  Walter  (65).  It  is  a  kind  of  hybrid 
system  with  nonbinary  features.  An  equivalent  binary 
codebook  representing  the  coding  modes  of  the  system  can 
be  found.  An  experimental  setup  for  the  evaluation  of 
variable  rate  satellite  communication  systems  with  PN 
sequences  has  been  designed  in  Canada  and  is  described  by 
Hedemark  (65).  Also  from  Canada  originated  a  brief 
comparison  between  analog  matched  filters  and  digital 
matched  filters  for  the  detection  of  PN  sequences  from 
lonosonde  transmitters  (Coll  et  al  65). 

Gold  (65)  wrote  a  paper  on  the  generation  of  large  families 
of  binary  sequences  with  specified  small  cross-correlation 
characteristics.  The  application  of  such  sequences  may 
be  found  in  spread  spectrum  multiplexing  systems.  Motley 
and  Melvin  (66)  described  a  higher  order  alphabet  (HOA 
equal  to  nonbinary)  system  using  a  biorthogonal  codebook 
of  8  dimensions  and  16  code  words  (n  =  4)  and  compared 
its  performance  over  high  frequency  (HF)  radio  channels 
with  conventional  diversity  systems.  In  1 966  also  appeared  a 
report  by  Gibson  (66)  on  the  performance  of  data  transmission 
systems  using  nonbinary  alphabets.  We  quoted  this  report 
earlier  but  we  would  like  to  stress  now  the  comprehensive 
treatment  of  logical  circuit  design  for  nonbinary  systems 
that  can  be  found  in  this  report.  This  may  be  useful  for 
design  engineers. 

Blasbalg  ct  al  (67)  used  pseudo-noise  binary  sequences 
in  a  conceptual  system  design  of  a  satellite  communications 
system  connecting  aircraft  via  the  satellite  to  other  aircraft 
or  to  ground  stations.  Barnes  et  al  (67)  described  the  data 
handling  system  in  the  ground  station  for  the  reception 
of  the  telemetry  signals  from  the  biorthogonal  8-bit-per- 
word  system  that  we  discussed  above  according  to  the  descrip¬ 
tion  of  Saliga  (67).  Andrews  (67)  reports  the  design  and 
field  tests  of  a  multipurpose  acoustic  telemetering  system 
that  uses  PN  sequences  of  different  lengths.  A  long  sequence 
serves  as  a  timing  reference  and  a  short  sequence  carries  data. 

Concluding  this  section,  we  may  state  that  the  most 
advanced  designs  of  information  transmission  systems  with 
multidimensional  codebooks  may  be  found  in  space  communi¬ 
cations  applications.  Evidently,  these  are  power  limited 
designs  and  therefore  operate  at  very  low  bit  density.  There 
is  a  large  number  of  other  conceptual  designs  and  of  some 
experimental  systems  where  designers  try  to  make  use  of 
PN  sequences  with  binary  elements.  Practically  no  reports 
are  available  of  experimental  efforts  in  the  area  of  codebooks 
with  nonbinary  elements  or  of  system  designs  of  multi¬ 
dimensional  systems  with  medium  or  high  bit  density. 
It  seems  that  the  practical  engineers  have  not  yet  discovered 
the  theoretical  conclusions  which  have  been  available  for  at 
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least  ten  years  and  which  clearly  prove  that  the  most  eco¬ 
nomical  application  of  multidimensional  systems  will  be  in 
connection  with  nonbinary  codebooks  (linear  real  coding) 
and  in  the  range  of  high  bit  densities.  Surely  the  reason 
for  this  lack  of  following  the  theoretical  lead  is  also  the  lack 


of  operational  coders  and  decoders  which  can  handle  largo 
codebooks  with  nonbinary  elements.  The  amount  of  cir¬ 
cuitry  required  for  this  task  will  turn  these  coding  devices 
into  special  purpose  computers.  Yet  it  seems  that  LSI 
will  make  such  feats  possible  in  the  near  future. 


Part  IV — Nonbinary  Signal  Transmission 


Introduction 

Part  III  contained  a  review  of  all  codebooks  for  block 
codes  with  binary  and  with  nonbinary  elements.  The 
mathematical  models  which  we  presented  in  Part  III  assumed 
that  an  ideal  transmission  system  would  be  available  for 
transferring  the  code  words  from  the  output  of  the  encoder 
to  the  input  of  the  decoder.  We  frequently  referred  to  the 
practical  difficulties  that  have  to  be  overcome  when  attempt¬ 
ing  to  come  close  to  the  specification  of  2B  samples  per  second 
used  in  such  ideal  transmission  systems. 

In  the  following  Part  IV  it  will  be  shown  that  there  is  a 
research  effort  under  way  which  parallels  the  research  on 
codebooks.  This  parallel  research  effort  is  known  under 
such  different  names  as  signal  theory,  waveform  research, 
research  on  bandlimited  transmission  functions,  and  others. 
It  is  closely  related  to,  but  not  identical  with,  modulation 
theory,  filter  theory,  and  spectral  analysis. 

M.  Fundamentals  of  Signal  Transmission  Modes 

Figure  16  in  Part  III  shows  that  in  multidimensional 
systems  the  encoder  delivers  1-tuples  of  real  numbers,  which 
cause  selections  in  the  modulator  matrix  from  a  waveform 
library  of  q  members.  In  special  cases  q  may  go  to  infinity ; 
such  special  cases  lead  to  libraries  of  waveforms  with  con¬ 
tinuously  variable  parameters.  If  q  is  a  finite  integer  We 
need  waveform  libraries  with  fixed  parameters. 

Ml.  The  Research  on  Waveform  Libraries  for  Multidimen¬ 
sional  Systems 

The  research  on  waveform  libraries  basically  follows  two 
schools  of  thought. 

The  research  effort  of  one  school  is  directed  to  the  explora- 
tion  of  linear  and  nonlinear  filters  (or  active  converters) 
with  capabilities  to  deliver  at  the  output  a  desirable  wave¬ 
form  library  if  the  individual  codebook  words  are  placed 
at  the  input  of  these  conversion  devices.  McAuliffe  (59) 
studied  the  impulsing  of  linear  networks  and  their  use  both 
as  signal  generator  in  data  transmitters  and  as  matched 
filters  in  data  receivers.  Dishal  (59)  investigated  in  particu¬ 
lar  the  impulsing  of  filters  with  Gaussian  response  shape. 
Eggers  and  Sze  (61)  reviewed  the  state-of-the-art  of  function 
generators,  as  they  are  used  in  analog  computing  technology, 
primarily  in  simidation  systems.  Many  of  these  devices 
operate  as  digital-to-analog  converters  rather  than  as  signal 
generators.  The  waveform  characteristics  are  completely 
determined  by  the  digital  input  ;  the  device  merely  shapes 
the  signals  to  deliver  the  desired  waveform  at  the  output. 
Billings  (62)  based  his  investigation  of  a  sequential  generat  ing 
filter  on  the  fact  that  the  output  of  a  linear  network  is  the 
convolution  of  the  input  signal  and  the  impulse  response 
of  the  network.  His  function  generator  delivers  an  approxi¬ 
mation  to  the  desired  impulse  response  every  time  it  is 
triggered,  and  its  total  output  is  the  sum  of  the  impulse 
responses  to  all  input  trigger  pulses.  Similar  devices  are 
reported  (Romanov  and  Zelentsov  64)  using  a  pulse  filter 
with  ferrite  cores,  and  (Fenwick  and  Barry  65)  producing  a 
linear  frequency  sweep  waveform  with  the  help  of  a  com¬ 
mercial  frequency  synthesizer  when  receiving  digital  input 
sequences.  A  rotational  transformation  of  signals  (Lang  65) 
is  an  analog  encoding  technique  which  will  be  of  interest  to 


codebook  specialists  and  to  waveform  specialists.  Related 
contributions  are  also  avadable  from  Hofheimer  and  Perry 
(58),  from  DeClaris  et  al  (61),  and  from  Fulford  (64). 

The  research  of  the  other  school  was  guided  by  the  necessity 
to  overcome  the  practical  limitations  of  transmitters  and 
channels  mentioned  in  Part  I,  subsections  Bi  and  B2. 
Many  of  the  contributors  mentioned  in  those  subsections 
also  participated  in  research  efforts  to  overcome  these 
limitations.  In  addition.  Tufts  (65)  derived  optimum 
transmitter  pulse  shapes  and  carried  out  the  joint  optimiza¬ 
tion  of  transmitter  and  receiver  in  detail  for  the  case  of 
signalling  through  a  noisy  RC  filter.  Although  investigation 
is  primarily  for  PAM,  it  is  also  of  fundamental  importance 
for  nonbinary  systems.  Pushman  (63)  calculated  the  spectral 
density  distributions  of  signals  for  binary  data  transmission 
under  rather  practical  assumptions. 

More  complex  waveforms  will  be  synthesized  from  ortho¬ 
gonal  components.  For  the  low-pass  channel  it  is  convenient 
to  use  sin  w0  t/w0  t  components  shifted  in  time  for  incre¬ 
ments  of  At  w/w „  ,  where  w«  =  2wf0  is  the  cutoff  frequency 
of  t  he  low-pass  channel  (ideally  assumed  to  have  a  rectangular 
frequency  response).  Zayezdnyy  and  Eydukyavichyus  (63) 
developed  a  system  of  abbreviated  representation  which 
they  demonstrate  with  the  help  of  numerical  examples. 
For  many  signals  of  length  T  it  is  possible  to  use  less  than 
2f«  T  component  waveforms,  if  a  new  smaller  set  of  com¬ 
ponent  waveforms  can  be  found  through  transformations 
of  the  sin  w„  t/w0  t  set  of  2f„  T  components  (f0  =  w„/2tt). 
The  two  Russian  authors  demonstrate  how  such  transforma¬ 
tions  can  be  found. 

Arthurs  and  Dym  (62)  prove  a  theorem  that  any  finite 
set  of  physically  realizable  waveforms  of  duration  T,  such  as 
s,  (t),  s2  (t),  .  .  .  ,  s  ,  (t),  may  be  expressed  as  a  linear  combina¬ 
tion  of  1  orthonormal  waveforms  <j>,  (t)  ;  <f>.,  (t)  ;  .  .  .  ;  (t) 

with  1  <[  q  : 

s,  (t)  =  a,,^!  (t.)  +  a, (t)  +  . .  .  +  a„^,  (t) 
s2  (t)  =  a,,<£,  (t.)  +  a «,</>.,  (t)  +  .  .  .  -|-  a21<£(  (t) 

‘  '  (93) 


%  (t)  —  a,,^I  (t)  +  (!)+•■•+  (t) 

where  the  an  are  real  numbers  given  by  the  formula  : 

a„  =  |%(t)  -Mt)dt  ,  (94) 

J  0 

with 

CT  0  if  i  #  j 

h  (t)  •  h  (t)cit  =  t ..  .  .  (95) 

j  0  1  if  1  =  j 

There  is  evident  similarity  between  thus  representation  of 
waveforms  by  orthonormal  component  functions  and  the 
representation  of  code  words  by  the  components  of  an 
1-dimensional  space.  This  apparently  is  the  strongest 
link  between  the  codobook  of  the  encoder  matrix  and  the 
waveform  library  of  the  modulator  matrix  in  the  basic 
block  diagram  of  a  nonbinary  system  shown  in  fig.  16. 
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Yet,  in  describing  systems,  there  are  not  many  publications 
which  make  full  use  of  this  similarity.  Part  II  explained 
how  this  general  waveform  library  can  be  specialized  to 
describe  the  one-dimensional  nonbinary  systems. 

M2.  The  Early  History  of  the  Waveform  Approach 

The  waveform  approach  originated  in  the  work  of  Nyquist 
published  in  1924  and  in  1928.  The  result  of  these  early 
investigations  was  the  general  recognition  of  the  Nyquist 
rate,  2B,  as  the  highest  possible  rate  at  which  independent 
elementary  signals  can  be  transmitted  without  mutual 
interference1  over  an  ideal  low-pass  channel  of  bandwidth  B. 
Nyquist’s  later  research  led  to  a  paper  (Nyquist  and  Pfleger 
40)  which  anticipates  the  later  highly  important  concept 
of  analytical  signals.  This  is  followed  by  Gabor’s  theory 
of  communications  (Gabor  4G)  that  preceded  Shannon’s 
mathematical  theory  of  communication  by  two  yoars. 
Gabor's  emphasis  was  more  on  deterministic  signals  and  on 
waveforms  ban  on  random  parameters  and  on  probability 
measures.  He  may  be  considered  as  the  father  of  the  signal 
theory  (waveform  approach)  as  much  as  Shannon  may  be 
considered  the  father  of  the  statistical  communication  theory 
that  led  to  the  coding  approach  of  the  1950’s.  Around  the 
middle  of  the  century  a  number  of  books  summarized  the 
state  of  the  fundamental  theories  of  communications  ;  they 
best  serve  in  toto  as  a  record  of  the  signal  theory  of  the  time  : 

(i)  1948  :  S.  Goldman,  “  Frequency  Analysis,  Modula¬ 
tion,  and  Noise  ”,  New  York,  McGraw-Hill  Book 
Co.,  a  theory  of  waveforms  and  modulated  signals 
on  the  basis  of  Fourier  series  and  Fourier  integrals. 

(ii)  1949:  K.  Kiipfmuller,  “Die  Systemtheorie  der 
elektrischen  Nachrichteniibertragung  ”,  Stuttgart,  S. 
Hirzll,  is  a  textbook  in  German  language  which 
approaches  the  signal  theory  from  the  well-established 
theory  of  transients  in  electrical  circuits. 

(iii)  1949  :  C.  Cherry,  “  Pulses  and  Transients  in  Com¬ 
munications  Circuits  ”,  London,  Chapman  and  Hall, 
Ltd.,  is  particularly  famous  for  the  extensive  treat¬ 
ment  of  the  spectra  of  pulsive  waveforms  and  of  the 
transient  analysis  of  networks  including  lines. 

(iv)  1950  :  J.  L.  Lawson  and  G.  E.  Uhlenbeck  (eds.), 
“  Threshold  Signals  ”,  Vol.  24  of  the  Radiation 
Laboratory  Series,  New  York,  McGraw-Hill  Book 
Co.,  is  now  considered  as  a  classical  textbook  for 
weak  signal  detection  theory.  It  approaches  signal 
theory  from  the  radar  engineer’s  point  of  view. 

(v)  1953  :  P.  M.  Woodward,  “  Probability  and  Informa¬ 
tion  Theory  with  Applications  to  Radar  ”,  London, 
Pergcmon  Press,  Ltd.,  discloses  many  wartime 
results  of  the  famous  Radar  and  Telecommunications 
Research  Establishment,  Malvern,  England.  Some 
of  the  mathematical  results  of  this  book  formed  the 
basis  of  extended  research  work  in  the  United  States. 

(vi)  1953  D.  A.  Bell,  “  Information  Theory  and  Its 
Engineering  Applications  ”,  explains  in  nonmathe- 
matical  language  how  the  early  signal  theory  of  Lord 
Kelvin,  Nyquist,  Hartley,  and  Gabor  may  be  integ¬ 
rated  into  the  more  modern  information  theory  of 
Wiener  and  Shannon. 

(vii)  1953  :  S.  Goldman,  “  Information  Theory  ”,  London, 
Constablo  and  Co.,  Ltd.,  has  an  introduction  to  the 
use  of  signal  space,  and  chapters  on  the  information 
theory  aspects  of  modulation  and  on  linear  correlation. 

Despite  this  impressive  array  of  textbooks,  the  waveform 
approach  to  t  he  conceptual  design  of  multidimensional  data 
transmission  systems  made  only  marginal  progress  prior 
to  1957.  Interesting  results  came  from  Sunde  (54)  and 
from  French  researchers  promoting  the  theory  of  analytical 
signals  (Ville  48,  Oswald  50  and  51).  The  French  researchers 
put  the  signal  theory  of  the  bandpass  channel  on  a  sound 
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mathematical  basis  by  combining  the  two  quadrature- 
phase  components  of  a  real  signal  into  a  complex  “  analytical 
signal  ”.  By  this  method  it  is  possible  to  apply  the  complex 
time  domain.  When  relating  this  domain  to  the  well- 
known  complex  frequency  domain,  one  can  apply  the  theory 
of  analytical  functions.  The  work  of  Sundo  combines 
signal  theory  with  circuit  theory  by  giving  the  general 
relations  between  frequency  response  and  pulse  transmission 
characteristics  in  the  form  of  a  compendium  for  engineering 
applications.  Some  of  the  general  results  of  signal  theory 
prior  to  1957  were  summarized  in  a  special  issue  of  “  IRE 
Transactions  of  Circuit  Theory  ”,  Dec.  1950,  with  an  intro¬ 
ductory  paper  by  Huggins  (56).  Although  few  of  these 
results  are  directly  applicable  to  waveform  alphabets  of 
multidimensional  data  systems,  many  of  the  papers  of  that 
special  issue  will  have  a  strong  indirect  influence  on  waveform 
alphabets  in  the  future. 

M3.  Progress  of  the  Waveform  Approach  Since  1956 

In  this  section  we  shall  review  only  those  contributions 
to  waveform  alphabets  that  are  of  a  general  nature.  The 
evolution  of  specific  sets  of  waveforms  will  be  discussed  in 
the  next  two  sections. 

The  special  signal  theory  issue  of  the  “  IRE  Trans,  on 
Circuit  Theory  ”  (Huggins  56)  marked  the  starting  point 
for  a  more  intensive  research  on  waveforms  for  data  trans¬ 
mission.  In  that  issue  we  find  an  article  by  Oswald  (56) 
extending  his  earlier  theory  of  analytical  signals  to  the  case 
of  bandlimited  signals  and  applying  this  extended  theory 
to  the  analysis  and  synthesis  of  waveforms  in  carrier  systems. 
While  the  older  signal  theory  of  carrier  systems,  using  the 
Fourier  integral,  relied  completely  on  the  study  of  the 
transmission  of  sinusoidal  waves,  Oswald  (56)  showed  that 
the  concept  of  analytical  signals  permits  the  extension  of  the 
signal  theory  to  cases  of  arbitrary  signals.  This  is  achieved 
through  the  introduction  of  a  quadrature  signal.  The 
extended  theory  defined  also  an  instantaneous  frequency  that 
played  an  essential  role  in  French  research  contributions 
on  message  intelligibility. 

Oswald’s  paper,  together  with  a  paper  by  Lampard  (56b), 
in  the  same  special  issue  gave  a  fundamental  discussion  of 
the  meaning  of  band  limitation  as  related  to  the  uncertainty 
principles.  Despite  the  existence  of  Gabor’s  definitions 
of  an  “  effective  temporal  duration  ”  and  an  “  effective 
spectral  width  ”  (Gabor  46),  a  stronger  expression  of  the  time- 
bandwidth  relation  was  felt  desirable  at  that  time.  (These, 
and  later,  results  are  discussed  in  the  next  subsection,  M4.) 

In  1958  several  authors  amplified  the  results  of  the  special 
signal  theory  issue  of  1956.  Silverman  (58)  pointed  out 
some  conditions  for  the  existence  of  moments  of  analytical 
signals  which  had  been  generally  overlooked.  Dugundji  (58) 
demonstrated  that  the  Hilbert  transform  of  a  waveform 
can  be  used  profitably  in  the  analysis  of  rather  arbitrary 
signals  which  had  been  introduced  by  Rice  (44)  as  multi- 
chromatic  waveforms.  Dugundji  (on  page  53)  gives  a 
formula  for  the  envelope  of  a  carrier-band  waveform  which 
is  “  much  simpler  than  Rice’s  formula  and  which  is  easier  to 
handle  analytically  ”.  Dugundji  uses  a  complex  valued 
function  called  the  pre-envelope  and  he  shows  that  Rice’s 
envelope  is  the  absolute  value  of  the  pre-envelope.  “  By 
using  tho  pre-envelope  the  envelope  of  the  output  of  a  linear 
filter  is  easily  calculated  and  this  is  usod  to  compute  the 
first  probability  density  for  tho  envelope  of  the  output  of  an 
arbitrary  linear  filter  when  the  input  is  an  arbitrary  signal 
plus  Gaussian  noise  ”. 

Also  in  1958,  Wernikoff  submitted  his  doctoral  thesis 
at  MIT  under  the  title  :  “  A  Theory  of  Signals  ”.  Wernikoff 
developed  a  formalism  for  the  analysis  of  signals  and  their 
interaction  with  linear  systems.  Signals  are  considered  as 
vectors  in  a  spaco  that  is  not  necessarily  finite  dimensional. 
Tho  space  can  span  ovor  an  orthogonal  basis  or  over  a  non- 
orthogonal  (but  linearly  independent)  set  of  vectors  as  basis. 
Wernikoff  describes  the  application  of  his  theory  of  signals 
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to  the  discrimination  of  individual  events  in  noisy  radar- 
like  systems  and  to  the  derivation  of  the  optimum  (in  the 
sense  of  least  peak  error)  predicting  filter.  Beyond  those 
examples  ono  can  see  applications  in  communications 
systems  with  signal  sets  which  are  synthesized  from  ortho¬ 
gonal  components. 

Another  major  contribution  to  the  research  on  signal 
transmission  inodes  is  more  on  the  practical  side  :  Blackman 
and  Tukey  (58)  contributed  a  major  paper  on  the  measure¬ 
ment  of  power  spectra  from  the  point  of  view  of  communica¬ 
tions  engineering.  This  paper,  in  two  parts,  has  since 
become  a  classic.  The  authors  successfully  combined  the 
ideas  of  transmission  theory  (problems  of  instrumentation, 
recording,  and  analysis)  with  the  ideas  of  statistical  estima¬ 
tion  theory.  This  combination  provides  the  insight  necessary 
for  a  sound  theory  of  data  acquisition  and  data  reduction. 
Such  a  theory  is  essential  for  the  optimum  extraction  of  the 
noisy  information-carrying  waveforms  in  a  receiver  of  a 
nonbinary  transmission  system. 

Several  auxiliary  methods  with  important  applications 
in  signal  transmission  systems  were  either  reviewed  or  addi¬ 
tionally  developed  in  the  years  1959  to  1902.  Helm  (59) 
devoted  a  fundamental  paper  to  the  Z-transform,  particularly 
to  its  use  in  real-time  systems  controlled  by  digital  computers. 
Such  problems  will  be  of  high  importance  for  the  operation 
of  multidimensional  information  transmission  systems  of 
high  dimensionality  (1  >  1000).  Fano  (61),  in  chapter  5.7, 
discusses  the  fundamentals  of  bandlimited  time  functions 
and  forms  a  bridge  between  the  coding  approach  and  the 
waveform  approach.  The  importance  of  the  Hilbert  trans¬ 
form  for  the  signal  transmission  theory  has  been  stressed 
above  in  connection  with  the  discussion  of  analytical  signals. 
Kuo  and  Freeny  (62)  reviewed  in  a  tutorial  paper  the  proper¬ 
ties  of  this  transform,  showed  its  application  for  single  side¬ 
band  modulation,  and  discussed  its  connection  with  the 
concept  of  the  pre-envelope.  Beutler  (61)  showed  that  the 
Hilbert  space  concept  could  lead  to  a  unified  approach  to 
sampling  theorems  for — in  the  wide  sense — stationary 
random  processes.  Papoulis  (62)  had  some  interesting 
comments  on  an  estimate  of  the  truncation  error  in  Fourier 
integrals.  Kurz  (62)  analyzed  an  interesting  method  for  the 
generation  of  a  waveform  alphabet.  He  called  his  method 
orthogonal  digit  coding.  The  signal  set  (average  power 
limited)  is  formed  from  weighted  sums  of  eigenfunctions 
generated  by  an  integral  equation  with  its  kernel  correspond¬ 
ing  to  the  inverse  Fourier  transform  of  the  noise  power 
density  spectrum.  Specific  performance  results  are  given 
for  several  orthogonal  digit  codes  (with  nonbinary  elements) 
when  the  demodulated  Gaussian  noise  power  density  spectrum 
increases  with  increasing  frequency.  Bedrosian  (62)  uses 
the  analytic  signal  concept  to  postulate  a  rather  general 
formulation  of  all  analog  modulation  methods.  Known 
types  of  modulation  are  readily  identified  as  special  cases. 
A  new  type  of  modulation  (SSB-FM),  compatible  with 
existing  FM  receivers,  resulted  from  this  investigation. 
Rubin  and  DiFranco  (68)  pointed  out  that  the  analytic 
signal  representation  of  wide-hand  carrier  signals  is  not 
completely  identical  with  the  conventional  representation, 
which  uses  a  variable  amplitude  term  and  a  variable  phase 
term  in  a  trigonometric  function.  The  difference  between 
the  two  representations  becomes  negligible  if  the  carrier- 
band  signals  are  bandlimited. 

In  1 963  we  not  ice  an  increasing  awareness  of  the  importance 
of  orthogonal  signal  sets,  either  to  be  used  directly  as  the 
waveform  alphabets  (Harmuth  63a),  or  to  be  used  as  coordi¬ 
nates  in  a  signal  space  of  high  dimensionality  (Harmuth  63b). 
Tung  (63)  contributed  a  short  note  to  the  analysis  of  such 
bandlimited  signal  sots,  restricting  his  investigation  to 
functions  that  could  be  generated  from  second-order  ordinary 
linear  differential  equations.  Tung  used  a  minimum  RMS 
difference  criterion  in  his  search  for  an  optimum  set  of  band- 
limited  functions  that  could  be  used  as  coordinates  for  the 
decomposition  of  any  arbitrary  time-function  limited  to 


the  same  band  as  the  set.  Dollard  (63)  provided  a  more 
general  theory  of  the  function  space  of  a  fixed  number  of 
dimensions  when  imposing  time  and  bandwidth  limitations. 
Dollard  discussed  orthogonal,  biorthogonal,  and  simplex 
configurations  in  terms  of  bandlimited  functions. 

The  analytical  treatment  of  sets  of  bandlimited  functions 
elicited  more  interest  in  1965,  with  papers  on  a  variety  of 
important  problems.  Nuttall  (65)  derived  expressions  for 
signal  sets  with  the  minimum  Gabor  bandwidth  when  t.ho 
signals  themselves  are  abruptly  time  limited.  Zakai  (65) 
extended  the  definition  of  bandlimited  functions  and  of 
bandlimited  random  processes  to  include  functions  and 
processes  which  do  not  possess  a  Fourier  integral  representa¬ 
tion.  The  sampling  theorem  for  such  functions  was  derived. 
Azar  (65),  in  a  paper  on  Z-transforms,  took  into  account  a 
finite  sampling  time  and  showed  how  to  make  practical 
use  of  it.  Yao  and  Thomas  (65)  investigated  the  error  that 
occurs  when  a  bandlimited  signal  is  represented  by  a  finite 
number  of  sampling  terms  in  place  of  the  theoretically 
necessary  infinite  number  of  terms.  Gonsalves  (65a)  proved 
the  existence  of  sets  of  real  functions  whoso  autocorrelation 
functions  are  identical  with  the  functions  themselves  when 
taken  for  positive  arguments.  Such  functions  are  called 
autocorrelation-invariant  functions.  Evidently,  they  will  bo 
of  great  importance  in  transmission  signal  analysis.  The 
Laguerre  and  Legendre  functions  of  the  first  kind  show 
this  property  (see  subsection  N2). 

Voelcker  (66a,  b)  took  the  first  step  toward  a  unified 
theory  of  modulation.  His  paper,  presented  in  two  parts, 
summarized  most  of  the  important  results  of  the  conventional 
modulation  theory  based  on  analog  sinusoidal  signals  (62 
references).  Voelcker  developed  equations  for  the  descrip¬ 
tion  of  simultaneously  phase-modulated  and  envelope- 
modulated  waveforms  which  are  also  bandlimited.  The 
new  concept  is  the  use  of  real  and  complex  zeros  of  the  wave. 
It  is  designated  as  the  concept  of  modulation  as  zero  manipula¬ 
tion.  Both  kinds  of  zeros  are  mathematically  interpreted 
in  terms  of  the  factorization  of  a  Fourier  series.  Examples 
illustrate  the  relationships  between  zeros,  spectra,  envelopes, 
and  phase  functions. 

The  impact  of  a  report  issued  in  1966  by  the  US  Depart¬ 
ment  of  Commerce  (66),  Telecommunication  Science  Panel 
of  the  Commerce  Technical  Advisory  Board,  will  be  felt 
during  the  next  two  decades.  The  report  concluded  (p.  39) 
that  “  The  United  States  has  presently  a  grossly  inadequate 
technical  program  of  improving  the  overall  effectiveness 
of  the  utilization  of  the  spectrum.”  One  consequence  of 
this  report  will  be  an  intensified  research  effort  in  signal 
theory,  with  the  aim  to  provide  strictly  bandlimited  trans¬ 
mission  signals  so  that  mutual  interference  botweon  adjacent 
transmission  bands  can  bo  significantly  reduced. 

Improved  tools  for  analyzing  and  synthesizing  band- 
limited  signals  are  provided  by  Papoulis  (67)  and  by  a 
new  computational  procedure  calk'd  the  Fast  Fourier  Trans¬ 
form  (Bogert  67). 

M4.  Fundamentals  of  Bandlimited  Waveforms 

Multidimensional  systems  need  transmission  facilities 
which  can  keep  the  coordinate  values  of  the  different  dimen¬ 
sions  perfectly  separated  during  the  transmission  processes. 
Obviously,  this  can  be  done  by  transmitting  the  dimensions 
in  time  sequence  (time  division)  or  simultaneously  in  different 
subchannels  (frequency  division).  It  has  been  known  for 
many  decades  that  it  is  impossible  to  limit  a  signal  simul¬ 
taneously  with  absolute  precision  in  the  time  domain  and 
the  frequency  domain.  Wozencraft  and  Jacobs  (65)  proved 
(in  Appendix  5B)  that  any  real-time  function  that  is  iden¬ 
tically  zero  over  any  interval  of  non-zero  length  has  an 
infinite  frequency  spectrum.  Conversely,  it  may  be  stated 
that  an  exactly  bandlimited  function  (i.e.,  a  function  with  a 
frequency  spectrum  that  is  identically  zero  outside  a  finite 
frequency  band)  must  have  an  identically  zero  value  over 
all  times  as  soon  as  it  is  identically  zero  over  any  finite  time 
interval. 
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This  fundamental  theorem  for  time-limited  or  bandlimited 
functions  caused  early  reasereh  workers  in  signal  theory  to 
search  for  j  a  quantitative  answer  to  the  question  for  the 
smallest  time-bandwidth  product  that  any  signal  of  finite 
energy  ever  can  assume.  Evidently  the  difficulty  is  in  the 
definition  of  an  “  effective  signal  duration  ”  and  “  effective 
signal  bandwidth  ”,  Contributions  by  Gabor  (40),  Woodward 
(51),  Lampard  (56b),  and  more  recently  by  Slepian  and 
Poliak  (01),  and  Landau  and  Poliak  (01  and  02)  led  to  results 
which  have  been  summarized  (and  supported  by  analytical 
proofs)  in  the  book  of  Wozencraft  and  Jacobs  (05)  (Appen¬ 
dix  5A)  as  follows  : 

There  are  two  theorems  on  the  time-bandwidth  (TB) 
product  of  time-limited  and  bandlimited  waveforms.  The 
one  theorem  is  due  to  Landau  and  Poliak  (62),  and  tho  other 
is  due  to  Shannon.  Both  theorems  prove  that  a  low-pass 
time  function  of  unit  energy  that  is  exactly  time-limited 
to  an  interval  —  T/2  <  t  <  +  T/2  and  that  has  not  more 
t  han  a  fraction  ( 0)  of  the  total  energy  outside  a  low-pass  band 
from  O  to  B  Hz  can  be  closely  approximated  by  a  set  of 
L  orthonormal  functions.  The  theorems  give  both  the  number 
L  and  the  closeness  of  approximation  according  to  an  RMS 
criterion.  Tho  first  theorem  is  the  stronger  one  ;  it  requires 
that  no  linear  combination  of  all  orthonormal  functions 
must  ever  have  more  than  a  fraction  ( 0)  of  the  onergy 
outside  the  |  frequency  band  O  to  B.  The  second  theorem 
(Shannon)  requires  that  merely  tho  orthonormal  functions 
themselves  must  meet  the  above  condition,  while  certain 
linear  combination  of  orthonormal  functions  could  have  a 
larger  fraction  than  0  outside  the  frequency  band  O  to  B. 
With  these  restraints  imposed,  the  following  formulas  may 
be  applied  : 

First  Theorem  :  (Landau  and  Poliak,  stronger  constraint)  : 

L  =  largost  integer  <  2TB  -f-  1  (96a) 

e^  =  12  0  (96b) 

Second  Theorem  :  (Shannon,  weaker  constraint)  : 

,  12  1 

L  —  largest  integer  <  2TB  -( - (1  -) - log,,  2TB)  (97a) 

A  7T2 
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for  nil  A,  0<  A  <  12 


(97b) 


In  both  caifes  the  error  of  the  approximation  (eA)  is  defined 
^is  the  energy  of  the  difference  between  the  low-pass  time 
function,  f(t),  and  its  approximation  : 
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Both  theorems  can  easily  be  expanded  to  bandpass  wave¬ 
forms.  Evidently  the  best  approximation  will  be  achieved 
by  that  set  of  orthonormal  functions  {</;,-  (t)}  which  can  supply 
the  largest  nilmbcr  of  functions  i  1,  2,  .  .  .  ,  q„  ,  meeting 
the  condition  of  the  band  limitation  (0  for  any  given  band¬ 
width  B).  Thus,  the  theorems  may  also  be  used  to  predict 
the  size  of  the  best  orthonormal  set.  of  a  given  TB  product 
when  meeting;  the  0  fraction  restraints  of  each  theorem. 

This  leads  Ip  the  following  equations  for  q„  : 

First  Theorem  :  (stronger  constraint)  : 

q„  <  L  <  2TB  +  1  (99) 

Second  Theorem  : 


q,  <  2TB  . 


Jl2  —  A 
12(  v)~^A 


r  12  / 
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(100) 


for  all  A,  0  <  A  <  12. 

For  T  ->oo  there  is  a  limiting  value  : 

lim  q„  2B 

—  <  - 

T  ->-*>  T  1  —  0 


(101) 


Equation  101  leads  to  the  well-known  Nyquist  rate  of 
independent  samples  (dimensions)  when  the  exactly  band- 
limited  sin  x  over  x  waveforms  for  which  0  =  0  are  used. 

For  a  very  large  TB  product  (for  example,  when  2TB  >  100) 
and  for  0  =  1/12,  equation  100  gives  the  frequently  used 
result  : 

q„<  2.4  TB  (102) 

The  above  equations  will  permit  future  comparisons  of 
the  bond  limitation  of  the  various  sets  of  orthonormal  wave¬ 
forms  that  have  been  suggested  in  the  last  decade.  Unfor¬ 
tunately,  for  many  of  these  sets,  0  has  not  been  computed 
yet  as  function  of  a  suitably  normalized  bandwidth. 

Contributions  closely  related  to  the  TB  problem  of  band- 
limited  waveforms  are  also  available  from  Petrich  (63), 
Dollard  (63),  Hofstetter  (64),  Tufts  and  Shnidman  (64), 
DiToro  and  Steiglitz  (65),  Jagerman  (65),  Nuttall  (65),  and 
Papoulis  (67). 


M5.  Concepts  of  Systems  with  Universal  Signal  Space 

There  have  been  numerous  efforts  to  combine  the  coding 
approach  and  the  waveform  approach  to  integrated  systems. 
An  integrated  system  may  be  defined  as  a  system  where 
each  of  the  separate  subsystems  is  first  optimized  according 
to  its  own  performance  criteria  and  where  easily  accessible 
operational  parameters  of  each  subsystem  are  modified 
until  the  total  system  reaches  its  optimum  performance 
under  the  externally  imposed  restraints  (cost,  peak  or  average 
power  limitations,  reliability,  serviceability,  etc.).  The 
different  functions  (subsystems)  in  a  communications  system 
that  may  be  subjected  to  such  an  optimization  process 
were  reviewed  as  early  as  1959  (Filipowsky  59).  Here  we 
are  concerned  only  with  the  integration  of  the  two  subsystems 
shown  in  fig.  16  :  the  digital  encoder  and  the  analog  wave¬ 
form  generator  (modulator).  The  optimal  combination  of 
coding  and  modulation  has  been  the  subject  of  several 
conceptual  expositions  during  the  time  prior  to  1960  (Costas 
52,  56,  59).  A  comprehensive  study  was  performed  at  tho 
Boll  Telephone  Laboratories  (Dollard  et  al  61  and  I.  Jacobs 
et  al  63),  summarizing  the  research  results  at  that  time, 
laying  the  groundwork  for  improved  system  designs,  and 
giving  an  equal  importance  to  coding  and  to  modulation 
methods.  Kirshner  (62)  reviewed  tho  theory  of  a  multi¬ 
dimensional  signal  space  for  the  purpose  of  increasing  the 
number  of  available  channels  by  allowing  a  certain  amount 
of  mutual  interference  between  the  channels.  Halsted  (63) 
applied  the  signal  space  concept  to  show  how  square  coding 
matrices  could  be  used  to  synthesize  optimum  signal  en¬ 
sembles  for  nonlinear  and  non-Gaussian  channels.  In  a 
similar  way  it  seems  possible  to  counteract  pulsive  noise  by  a 
rotational  transformation  of  signals  (Lang  63).  This  concept 
is  related  to  the  smearing  techniques  of  continuous  systems 
and  its  application  allows  interesting  explorations  of  tho 
characteristics  of  special  codes.  This  is  particularly  true 
when  applying  nonorthogonal  transformations  in  place  of 
tho  pure  rotational  transformation.  Dollard  (63)  developed 
t  he  concept  of  a  universal  signal  space  on  a  mathematically 
rigorous  basis  and  Amari  (65)  penetrated  more  deeply  into 
the  metrics  of  signal  spaces.  Ho  defined  the  entropy  of  a 
signal  space  and  explored  the  problem  of  continuous  mapping 
between  signal  spaces  of  different  dimensions. 

H.  D.  Luke  (65)  continued  the  approach  which  had  been 
initiated  by  McAuliffe  (59),  applying  the  later  results  of 
signal  theory.  By  generating  special  pulse  sequences  in 
the  encoder,  Luke  generated  a  suitable  waveform  alphabet 
in  transversal  filters.  This  method  is  a  practical  approach 
toward  a  system  with  universal  signal  space.  Luke’s 
results  are  related  to  an  effort  by  S.  H.  Chang  (66a)  (section 
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K2).  Chang  is  primarily  interested  in  the  generation  of 
nonbinary  orthogonal  codes  with  integer  elements  ;  that 
part  is  related  to  Luke’s  approach.  Chang,  however,  does 
not  use  the  output  of  the  encoder  as  the  input  sequenco 
for  pulsing  a  network  to  generate  a  waveform. 

An  important  stepping  stone  to  the  further  evolution  of 
the  network  pulsing  techniques  is  the  investigation  of  the 
frequency  spectra  of  various  sets  of  binary  and  nonbinary 
sequences.  Gorog  (68)  performed  such  an  investigation 
on  various  redundant  codes.  When  also  incorporating  the 
channel  characteristics,  primarily  the  frequency  response, 
into  this  integrated  optimization  process  of  codes  and  wave 
shaping  devices,  one  follows  closely  the  conceptual  approach 
which  Costas  suggested  in  1952.  A  more  recent  study  in 
this  direction  is  due  to  Hancock  and  Quincy  (66). 

Possibly  the  most  fundamental  contribution  to  the  theory 
of  a  universal  signal  space  is  the  paper  by  Wyner.  (66b) 
about  the  capacity  of  the  bandlimited  Gaussian  channel. 
Wyner  claims  that  the  famous  channel  capacity  formula 
of  Shannon  (equations  17  to  23  and  curve  1  in  most  of  the 
utility  charts  of  this  paper)  never  has  been  justified  by  a 
coding  theorem  (and  converse).  Such  a  theorem  is  necessary 
to  establish  Shannon’s  channel  capacity  as  the  supreme 
expression  for  the  capacity  of  more  restrictive  mathematical 
models.  Wyner  proposes  a  number  of  physically  consistent 
models  and  establishes  equation  17  as  the  capacity  for  each 
of  the  models. 

N.  Special  Waveform  Alphabets 

Nl.  Monary  and  Binary  Large  TB  Waveforms 

Radar  technology  was  the  first  to  make  extensive  use 
of  the  matched  filter  technique  as  originally  suggested  by 
Wiener  (49)  and  Woodward  (53).  This  matched  filter 
technique  is  an  optimum  method  to  extract  a  known  signal 
from  noise.  The  technique  found  conceptual  acceptance  in 
communications  with  the  first  mathematical  model  of  an 
ideal  binary  system  (Reiger  53).  One  of  the  first  practical 
matched  filter  extractors  was  applied  in  the  RAKE  system 
(Price  and  Green  58,  section  L3).  Sussman  (60)  describes 
the  matched  filter  action  in  that  system. 

Evidently  the  transmission  waveforms  which  had  been 
selected  as  large  TB  waveforms  for  the  first  binary  matched 
filter  communications  systems  were  a  good  basis  for  the 
design  of  larger  waveform  alphabets.  The  use  of  such  larger 
alphabets  was  clearly  suggested  in  the  interest  of  trans¬ 
mitting  more  information  (nonbinary  systems),  of  higher 
security  (code  switching),  or  for  the  purpose  of  addressing  a 
larger  number  of  receivers  (multiple  access  systems). 

In  this  connection  we  find  that  Key  et  al  (61)  discussed  a 
method  for  designing  large  TB  signals  with  independently 
specified  envelope  and  autocorrelation  function .  Gretten- 
berg  (63)  proposed  a  signal  selection  procedure  based  on  a 
maximum  divergence  criterion  which  requires  that  the  smallest 
value  of  the  divergence  between  two  distinct  messages  be 
maximized.  Kadota  (64)  wrote  about  systems  with  random 
signals  having  large  TB  products  and  known  covariance 
functions.  Such  systems  constitute  a  mathematical  idealiza¬ 
tion  of  some  radio  communication  links  containing  a  random 
medium. 

A  particularly  interesting  class  of  large  TB  signals  is  the 
swept  frequency  signals,  also  called  chirp  signals  because 
of  their  similarity  with  the  audio  waveform  of  a  siren  sound. 
Such  signals  change  their  instantaneous  frequency,  usually 
in  a  linear  manner,  during  the  signal  duration.  A  binary 
system  can,  of  course,  use  one  waveform  starting  with  the 
lowest  frequency  in  the  band  and  ending  with  the  highest 
frequency  and  another  waveform  following  the  opposite 
pattern.  When  using  more  complex  patterns  of  frequency 
variation,  larger  sets  of  waveforms  may  be  designed.  Signals 
of  this  kind  have  been  used  in  radar  systems  for  some  time, 
but  their  application  in  communications  is  of  more  recent 
date.  Holland -Moritz  et  al  (66)  described  a  communications 


system  which  uses  swept  frequency  modulation,  with  the 
individual  large  TB  waveforms  overlapping  in  the  time 
domain.  This  feature  makes  the  system  a  polysignal 
system,  following  the  definitions  used  in  this  paper.  Such 
systems  are  beyond  the  scope  of  the  present  paper  ;  yet 
the  description  of  t  he  waveforms,  of  their  modes  of  generation, 
and  of  the  extraction  devices  may  be  of  interest  here.  The 
description  of  another  communications  system  with  FM 
chirp  signals  is  due  to  Griffiths  and  Smith  (67). 

N2.  Polynomial  Waveform  Alphabets 

The  following  review  will  show  that  there  is  much  interest 
in  orthogonal  polynomials  and  in  their  use  as  waveform 
alphabets,  primarily  as  orthonormal  alphabets.  Yet  it  is 
amazing  that  the  mathematical  foundations  for  such  efforts 
are  at  least  one  century  old.  The  particular  orthogonal 
polynomials  that  gained  most  of  the  interest  of  communica¬ 
tions  engineers  are  (in  the  order  of  their  frequency  of  applica¬ 
tion  in  the  communications  literature)  : 

(i)  The  Legendre  Polynomials.  Andrien  Marie  Legendre 
(1752-1833),  a  French  mathematician,  is  noted  for 
his  work  on  the  theory  of  numbers  (‘'  Theorie  des 
Nombres  ”,  1830)  and  for  his  extensive  study  of 
elliptic  integrals  (“  Traite  des  Functions  Elliptiques  ”, 
3  vols.,  1825-1832). 

(ii)  The  Laguerre  Polynomials.  Edmond  Nielas  Laguerre 
(1834-1886),  a  French  mathematician,  is  famous  for 
his  contributions  to  modern  geometry.  His  collected 
works  were  published  in  1898. 

(iii)  The  Hcrmite  Polynomials.  Charles  Hermite  (1822- 
1901),  a  French  mathematician,  made  valuable 
contributions  to  the  theory  of  numbers  and  t  he  theory 
of  elliptic  functions. 

(iv)  The  Jacobi  Polynomials.  Karl  Gustav  Jakob  Jacobi 
(1804-1851),  German  mathematician,  is  noted  for  his 
work  on  elliptic  functions  described  in  his  “  Funda¬ 
mental  Nova  Theoriao  Functionum  Ellipticarurn  ” 
(1829)  and  for  his  contributions  to  the  theory  of 
numbers. 

Orthogonal  polynomials  are  sets  of  functions  [f„  (x)]  of 
degree  n  defined  on  the  interval  a  <  x  <  b  with  respect 
to  the  weight  function  w(x).  Their  orthogonality  property 
can  bo  expressed  in  the  following  form  (Abramowitz  and 
Stegun  64)  : 

”  w(x)f„  (x)  .  fm  (x)dx  =  0  (103) 

a 

(n  ^  in  ;  n,  m  =  0,  1,  2,  .  .  .) 

The  weight  function  co(x)  has  non-negative  values  for  all 
x  values.  The  polynomials  f j  (x)  need  to  be  “  standardized  ”  ; 
i.e.,  they  have  to  meet  the  condition  of  equation  104  so  that 
(  hoy  may  be  in  the  correct  form  to  satisfy  a  number  of  relation¬ 
ships  that  apply  to  all  the  orthogonal  polynomials  mentioned 
above  and  to  many  other  kinds  of  orthogonal  polynomials 
which  may  have  potential  applications  in  communications. 

f  u>(x)ff  (x)dx  =  h„  (104) 

**  a 

fn  (x)  =  k„  x"  +  k;  X"-1  -f.  .  . 

(n  =  0,  1,  2,  .  .  .) 

The  relationships  that  all  the  classes  of  orthogonal  poly¬ 
nomials  must  satisfy  are  listed  below. 

(i)  All  polynomials  are  solutions  of  the  general  second 
order  differential  equation  : 

g2  (x)r  +  g,  (x)f;  +  g„  (n)r„  =  o  ,  (ios) 

where  g.>  (x)  and  g,  (x)  are  independent  of  n  but  depend 
on  the  class  of  polynomials.  g0  (n)  depends  only  on 
n  and  not  on  x,  but  differs  for  the  various  classes 
of  polynomials.  Abramowitz  and  Stegun  (64)  pub- 
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lished  tables  of  these  functions  for  all  orthogonal 
polynomials.  Table  1  gives  these  functions  and  k„ 
(equation  106a)  and  h„  (equations  104,  10(ia)  for  the 
polynomials  of  interest  in  connection  with  the  present 
paper. 

(ii)  All  polynomials  satisfy  the  following  recurrence 
relation  : 


f»+i  (x)  =  K  +  b„  x)  .  f„  (x)  -  c„  f._,  (x)  ; 

L  K  +  I  ,  /k„+1  k„\ 

°"  =  Hk - irj; 

Kn  VKn  +  l 

kn4i  •  k„_,  .  h„ 


k2 .  h„_, 

(iii)  All  polynomials  satisfy  the  Rodrigues  formula  : 


(100) 


(100a) 


f.  (x)  = 


1 


d" 


(n)  •  P<x)  dx” 


(p(x)  .  [g(x)l«} 


(107) 


The  functions  a„  (n),  p(x),  and  g(x)  are  listed  in 
Abramowitz  and  Stegun  (04)  for  each  class  of  ortho¬ 
gonal  polynomials.  Table  2  gives  these  functions 
for  the  four  classes  of  orthogonal  polynomials  of 
special  interest  in  the  present  paper. 


(iv)  For  some  of  the  polynomials  there  are  representations 
in  terms  of  hypergeometric  functions  : 

f„  (x)  =  d„  .  F(a  ;  b  ;  c  ;  x(x))  (108) 

Table  2  gives  the  special  values  for  d„  ,  a,  b,  c,  and  k(x) 
for  two  classes  of  orthogonal  polynomials  of  interest 
in  this  paper. 

(v)  There  are  many  important  interrelations  between 
orthogonal  polynomials  of  the  same  class.  The  reader 
is  referred  to  Abramowitz  and  Stegun  (64),  to  Jonsone 
(59),  to  Gernimus  (01),  and  to  Boas,  Jr.  and  Buck  (64) 
for  further  details. 

In  many  publications,  reference  is  made  to  orthogonal 
functions,  though  there  is  no  well-defined  terminology 
clarifying  the  difference  between  orthogonal  polynomials 
and  orthogonal  functions.  Possibly  the  best  explanation 
for  electronic  engineers  is  given  in  Y.  W.  Lee  (60)  who 
describes  in  full  detail  the  orthonormalization  process  for 
Laguerro  functions  and  Legendre  functions  when  starting 
with  the  respective  orthogonal  polynomials  described  in 
Tables  1  and  2.  From  this  and  similar  presentations  one 
may  conclude  that  the  general  practice  is  to  call  the  product 
of  an  orthogonal  polynomial  with  the  root  of  the  weighting 
function,  the  orthogonal  function  of  the  particular  class. 
In  addition,  Y.  \V.  Lee  (60)  shows  how  to  apply  orthogonal 
polynomials  and  orthogonal  functions  in  the  complex  domain. 


Table  1 

Coefficient  functions  and  other  parameters  for  various  classes  of  orthogonal  polynomials 


Class  of 
Polynomials 


Symbol 


g2  (x) 


g,  (x) 


go  (n) 


Legendre 


P„  <x) 


1  -  x2 


n(n  +  1) 


(— n)  ! 
2"(n  !)2 


2n  +  1 


Laguerre 


Hormite 


L„  (x) 


1  -  x 


H„  (x)  1 


—  2x 


(-1)" 
n  ! 


Vor  .  2*  .  n  1 


Jacobi 


P<«,  f)  (x) 


1  -  x2 


P  -  a  —  (a  +  p  +  2)x 


n(n  +  a  -f  +  1) 


1  /2n  -f  a  +  |3\ 
2"  V  n  ) 


2a+/5+i 


2n  +  a  +  ft  +  1 
r(n  +  a  +  l).r(n  +  /3  +  1) 
n  !P(n  +  a  +  /3  +  1) 


Table  2 

Coefficient  functions  and  other  parameters  for  the  Rodrigues’  formula  and  the  hypergeoinetric  function  when  representing  various  classes 

of  orthogonal  polynomials 


Class  of 

Polynomial 

Symbol 

a*  (n) 

p(x) 

d„ 

a 

b 

c 

x(x) 

Legendre 

— 

R,  (x) 

(-1)"  .  2"  .  n  ! 

1 

1 

— n 

n  +  1 

1 

1  -  X 

2 

Laguerre 

L„  (x) 

n  ! 

e-* 

X 

— 

— 

— 

— 

— 

Hormite 

H„  (x) 

(-D- 

e-*3 

e 

1 

— 

— 

— 

— 

— 

Jacobi 

Pt*.  fi)  (x) 

(1  -x)«.(l  +X)fi 

l  —  X2 

— n 

a  +  1 

1  -X 

2 
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The  use  of  orthogonal  polynomials  and  functions  for  signal 
analysis  was  recommended  by  Park  and  Glaser  (58).  That 
report  also  contains  the  construction  of  a  laboratory  model 
of  a  signal  analyzer  and  deals  with  Legendre  polynomials, 
Laguerre  functions,  and  Hermite  functions.  Forsyth  (57) 
and  Crouse  (63)  applied  orthogonal  polynomials  to  the 
problem  of  data  fitting  (with  a  digital  computer)  and  to  curve 
fitting  respectively.  Gonsalves  (64)  showed  that  both 
Laguerre  and  Legendro  functions  of  the  first  kind  have  the 
AI  (autocorrelation  invariant)  property.  Lipkin  (66)  demon¬ 
strated  that  Russian  scientists  aro  well  aware  of  the  potential 
of  Laguerre  and  Legendre  polynomials  for  applications  in 
communications  waveform  alphabets.  The  Rand  Corpora¬ 
tion  investigated  the  computational  techniques  resulting 
from  the  application  of  polymomials  in  airborne  radar  signal 
analysis  (Liechtenstein  66). 

Turning  now  to  applications  of  specific  orthogonal  poly¬ 
nomials,  we  recognize  that  Legendre  polynomials  are  fre¬ 
quently  suggested  for  signal  transmission  and  for  signal 
processing  applications.  Ballard  (62a)  described  an  experi¬ 
mental  design  of  a  four-channel  multiplex  system  using 
orthogonal  Legendre  polynomials  as  subcarrior  waveforms. 
The  system  became  known  as  the  Orthomux  telemetry  system. 
Ballard  (62b)  applied  tho  same  technique  of  orthogonal 
Legendre  polynomials  to  a  real-time  function  analyzer- 
synthesizer.  Garfinkel  (64)  published  a  new  addition  theorem 
for  the  derivative  of  a  Legendre  polynomial.  Duckworth 
and  Smith  (65)  discussed  a  problem  with  the  generating 
functions  of  Legendre  polynomials.  Broome  and  Dean  (64) 
presented  a  method  for  encoding  complicated  transient 
signals  of  seismic  P-waves  into  a  few  numbers  with  the  help 
of  sets  of  orthonormal  polynomials. 

Laguerre  polynomials  or  Laguerre  functions  were  suggested 
by  Kantz  (54)  for  an  improved  method  of  transient  synthesis, 
a  method  that  was  also  used  in  1964  for  the  analysis  of 
seismic  waves  by  Broome  and  Dean  (64)  as  reported  above. 
Some  recent  mathematical  contributions  to  the  theory  of 
Laguerre  polynomials  are  due  to  Head  (56),  Mishkin  (59), 
Gillis  and  Weiss  (60),  Samsononko  and  Ramancnko  (63) 
and  Clowes  (65).  Tables  of  Laguerre  polynomials  and 
functions  are  available  by  Flinn  and  Dean  (64)  and  by' 
Aizenshtadt  et  al  (66).  Applications  of  Laguorre  poly¬ 
nomials  are  reported  for  experimental  seismic  equipment 
(Broome  and  Dean  (64)  and  Dean  (64))  and  also  for  signal 
sorters  (Mattachione  (65),  Gonsalves  (65b)  and  correlators 
(Schetzen  (64),  Eier  and  Weinriehter  (68)). 

Hermite  polynomials  and  their  history  were  reviewed  by' 
Cameron  and  Martin  (47).  They'  were  tabulated  by  Berlyand 
et  al  (62).  They  were  used  in  an  experimental  orthogonal 
multiplexed  communications  system  (Karp  and  Higuchi 
63,  Karp  and  Rampacek  64).  The  modified  system  has  no 
discontinuities  in  the  waveform  and  has  a  small  TB  product. 

Jacobi  polynomials  were  used  by'  Armstrong  (57  and  59) 
for  the  representation  of  transients  in  an  orthogonal  system. 

Cliebyshev  polynomials  are  another  kind  of  orthogonal 
polynomials  well-known  in  filter  theory  but  so  far  seldom 
used  as  waveform  alphabets.  Kuly'a  (63  and  64)  mentioned 
their  use  in  an  experimental  vocoder  sy'stem  in  Russia. 
C.  S.  Chang  (66)  submitted  a  proof  about  the  discrete  ortho¬ 
gonal  property  of  Cheby’shcv  polynomials  which  made  them 
attractive  in  curve-fitting  applications. 

Concluding  the  section  on  polymomial  waveform  alphabets, 
we  may'  state  that  orthogonal  polymomials,  particularly 
when  used  in  modified  form  with  optimal  weighting  functions, 
have  a  good  potential  for  applications  in  nonbinary  com¬ 
munications  systems.  Yet  up  to  this  time  their  applications 
were  primarily  in  circuit  analysis,  for  the  decomposition  of 
transients,  and  in  some  multiplex  systems.  No  report,  to 
our  knowledge,  has  come  of  an  experimental  nonbinary 
monosignal  system  using  such  sets  of  waveforms.  A  com¬ 
parative  analysis  of  their  bandwidth  occupancy  would  bo 
necessary  before  the  relative  merits  of  the  various  classes 
can  be  evaluated  quantatively. 


N3.  Trigonometric  and  Exponential  Waveform  Alphabets 

Huggins  (56)  gave  a  comprehensive  review  of  the  state 
of  the  signal  theory  at  that  time.  Many'  readers  who  are 
not  familiar  with  the  special  signal  theory  issue  of  the  “  IRE 
Transactions  on  Circuit  Theory  ”  (Dec.  1956)  will  be  interested 
in  the  foresight  of  tho  early  research  workers  in  this  special 
branch.  It  seems  to  this  author  that  many  excellent  ideas 
of  the  time  prior  to  1956  have  not  been  sufficiently'  exploited, 
even  now,  twelve  years  after  their  publication.  One  idea 
was  the  exponential  waveform  alphabets.  Huggins  (56) 
stressed  their  fundamental  character.  The  orthogonal 
polynomials  discussed  in  the  previous  subsection  N2  use 
exponential  weighting  functions,  thereby  becoming  carriers 
for  exponential  waveforms.  Tho  trigonometric  product 
waveforms  to  be  discussed  in  the  next  section  surely  must 
be  related  to  exponential  waveforms  with  complex  exponents, 
although  this  aspect  is  still  a  subject  of  future  exploration. 

Clearly'  recognizing  the  central  importance  of  exponential 
waveform  alphabets,  Huggins  and  his  school  of  thought 
performed  wide-ranging  research.  Huggins  (63)  summarized 
seven  years  of  research  at  Johns  Hopkins  University  on  the 
representation  and  analysis  of  signals.  This  summary 
reproduces  reprints  of  several  publications  which  document 
the  research  results,  and  includes  abstracts  of  all  technical 
reports  produced  under  the  program.  Exponential  waveform 
alphabets,  primarily  sets  of  orthonormal  exponentials  are  at 
tho  centre  of  all  these  studies.  Ross  (62)  reviewed  the  most 
important  characteristics  of  orthonormal  exponentials  and 
quoted  four  doctoral  theses  which  investigated  related 
subjects.  Lory  et  al  (59),  Young  and  Huggins  (62a  and  62b) 
and  McDonough  (63)  are  among  tho  particularly  characteris¬ 
tic  publications  of  this  group  of  researchers. 

Outside  the  research  group  of  Johns  Hopkins  University 
we  find  interest  in  damped  oscillatory  exponentials  (Doiansky 
60)  as  base  signals  in  speech  signal  analysis.  Mondel  (63) 
studied  the  problem  of  characterizing  overdamped  systems 
from  data  in  the  time  domain  by  decomposition  of  the  data 
with  the  help  of  exponential  base  functions.  Tho  results 
of  this  study  were  further  extended  to  a  signal  approximation 
with  non-uniformly  weighted  orthonormal  overdamped 
exponentials  as  base  vectors  (Mendel  64a).  Mendel  (64b) 
also  extondod  Armstrong’s  earlier  work  (57  and  59)  by 
investigating  the  inversion  of  Laplace  transforms  by  means 
of  truncated  series  of  orthonormal  exponential  functions. 

Trigonometric  functions  as  sets  of  orthogonal  functions 
aro  actually  a  special  case  of  undamped  exponentials  with  an 
imaginary  exponent.  To  be  usefid  as  waveform  alphabets 
in  nonbinary  systems,  the  signals  have  to  be  restricted  to  a 
finite  TB  product,  i.e.,  to  a  given  signal  base.  Harmuth 
(60a  and  60b)  explored  the  possibility  of  using  harmonically 
related  sine  and  cosine  functions  (restricted  to  integer 
numbers  of  periods)  as  orthogonal  signal  sets  for  radio 
communications  and  for  communications  over  telephone 
circuits.  Franco  and  Lachs  (61)  used  orthogonal  sets  of 
trigonometric  functions  in  a  polysignal  waveform  library  as 
base  vectors  and  in  a  special  conceptual  system.  These 
orthogonal  sots  restrict  tho  set  of  quantized  vector  components 
to  the  ternary  set  +1,  0,  —1.  Tho  total  library  of  output 
words  is  an  equi-energy  waveform  library.  The  Franco 
and  Lachs  (61)  paper,  dealing  with  polysignal  systems,  is 
of  interest  because  it  uses  an  orthonormal  set  of  trigono¬ 
metric  functions.  Contrary  to  the  Franco  paper,  Aronstoin 
(63)  compared  a  single  signal  waveform  alphabet  of  the 
Harmuth  (60c)  variety  with  binary  block  codes,  arriving  at 
the  conclusion  that  trigonometric  waveform  alphabets  are 
superior.  Kirshner  (62),  in  his  frequently  mentioned  paper, 
likewise  uses  an  orthogonal  set  of  trigonometric  waveforms 
as  an  example.  The  research  of  Harmuth  (60a,  b,  c)  and 
Franco  and  Lachs  (61)  lead  to  tho  construction  of  a  proto¬ 
type  data  transmission  equipment  for  field  evaluat  ion  of  real 
time  data  transmission  over  radio  frequency  circuits  known 
as  the  DEFT  system  (Luke  64).  Sets  of  trigonometric 
waveforms  are  also  used  in  a  spectrum  analyzer  receiver 
described  by  Charles  and  Springett  (67).  Fundamental 
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ideas  along  similar  lines  had  been  published  as  early  as 
11)58  in  a  paper  of  Fourier  orthogonal  fillers  (Blasbalg  58). 

The  Onhomatch  Data  Transmission  System  uses  an  en¬ 
velope  orthogonal  set  of  trigonometric  waveforms.  We 
discussed  this  system  in  Part  II,  section  G  (Kuhn  et  al  (63)) 
and  compared  it  with  a  typical  PAM-FM  system.  In  this 
section  it  Will  be  of  interest  because  of  its  waveform  alphabet. 

In  section  O  it  will  become  evident  that  the  trigonometric 
waveform  alphabets  discussed  above  have  recently  been 
identified  las  the  first  order  of  a  much  larger  family  of  tri¬ 
gonometric  product  waveforms  consisting  in  general  of  more 
than  one  trigonometric  factor.  The  single  factor  waveforms 
which  have  been  discussed  here  have  the  disadvantage  that 
a  large  percentage  of  all  waveforms  display  discontinuities 
at  the  limits  of  the  time  interval,  thus  causing  an  unneces¬ 
sarily  large  TB  product. 

N4.  Perfectly  Bandlimited  Waveforms 

Shannon  (48)  used  a  perfectly  bandlimited  waveform, 
i.e.,  the  waveform  with  a  rectangular  energy  density  spect  rum, 
as  the  basis  for  his  fundamental  channel  capacity  theorem. 
This  waveform  has  in  the  time  domain  a,  function  of  the  class 
sin  x  over  Jk.  Numerous  authors  before  and  after  Shannon’s 
classical  papers  used  the  orthogonal  set  of  sin  x  over  x 
functions  shifted  for  a  time  interval  of  AT  =  1/2B  against 
each  other  as  the  classical  set  of  bandlimited  orthogonal 
waveforms.  Numerous  versions  of  sampling  theorems  are 
based  on  t  he  same  set  of  orthogonal  functions  and  the  values 
of  this  function  are  tabulated  in  most  collections  of  elementary 
functions.  In  Part  I  we  introduced  the  signal  base  /3  as 
the  TB  product  for  waveforms  that  do  not  interfere  with 
each  other!  We  recognized  that  pc  =  0.5  is  the  minimum 
value  of  this  signal  base.  This  minimum  value  again  is 
based  on  the  sin  x  over  x  function  with  a  rectangular  energy 
density  spectrum  ranging  from  zero  to  B  Hertz  and  being 
perfectly  zero  for  any  frequency  larger  than  B. 

The  problem  is  more  complex  when  considering  a  carrier- 
band  channel,  but  pairs  of  quadrature  signals  or  the  concept 
of  analytical  elementary  signals  have  solved  this  problem 
satisfactorily  (Giovanni  65).  The  problem  really  becomes 
crucial  when  attempting  to  impose  an  absolute  limit  to  the 
signal  energy  simultaneously  in  the  frequency  and  the  time 
domain.  We  recognized  in  subsection  M4  that  this  is  an 
impossible  condition,  violating  the  uncertainty  law,  and  that 
it  is  therefore  necessary  to  specify,  together  with  time  and 
frequency  limits,  a  percentage  of  the  energy  that  can  be 
tolerated  outside  those  limits.  Although  the  energy  outside 
the  frequency  band  B  is  perfectly  zero  for  sin  x  over  x 
waveforms,  one  will  quickly  recognize  that  the  energy 
outside  the  time  limit  T  =  1/2B  is  significantly  larger  than 
for  any  other  pulsive  waveform.  Indeed,  the  time  function 
of  a  sin  x  |  over  x  waveform  reaches  from  —  co  to  +  oo. 
It  is  therefore  essential  to  truncate  the  function  in  the  time 
domain  (Yao  65),  Bollard  (63),  a  process  which  in  turn 
disturbs  the  ideal  rectangular  frequency  spectrum.  The 
search  is  thus  initiated  for  the  kind  of  waveform  which  has 
the  best  overall  concentration  of  its  energy  into  a  given  TB 
product  with  a  minimum  of  the  energy  outside  the  limits 
of  the  signal  base  j}c  .  Many  indications  are  that  an  equal 
spread  of  the  energy  in  both  domains  is  the  best  compromise, 
and  this  leat|s  to  bell-shaped  (or  Gaussian)  signals  (Gabor  4(5 ) . 

In  this  subsection,  however,  we  are  concerned  with  perfectly 
bandlimited  jsignals.  The  recent  research  on  prolate  spheroidal 
wavefunctioms  (PS IF)  is  particularly  interesting  in  this  con¬ 
nection.  Slepian  and  Poliak  (61)  described  a  complete  set 
of  bandlimited  functions  which  possess  the  curious  property 
of  being  orthogonal  over  a  given  finite  interval  as  well  as 
over  the  interval  from  —  co  to  -f  oo.  These  functions  are 
eigenfunctions  of  the  finite  Fourier  transform.  Their 
properties  nmko  them  ideally  suited  for  the  study  of  certain 
[questions  regarding  the  relationship  between  functions  and 
|  their  Fourier  transforms. 
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The  application  of  the  theory  of  the  PSW  to  a  number  of 
questions  about  timelimited  and  bandlimited  signals  is 
presented  by  Landau  and  Poliak  (61).  In  particular  the 
authors  found  the  possible  percentage  of  the  energy  of  a 
finite  energy  signal  in  a  finite  time  interval  and  a  finite 
frequency  band.  These  authors  also  applied  the  theory  of 
PSW  to  the  search  for  signals  which  do  the  best  job  of 
simultaneous  time  and  frequency  concentration  of  their 
energy.  In  the  area  of  data  transmission  the  authors  (on 
p.  83)  are  “  interested  in  minimizing  both  the  tail  of  a  pulse 
outside  its  time  slot  and  its  spectrum  outside  of  an  assigned 
frequency  band  ”,  recognizing  that  it  is  not  possible  to  make 
both  these  “  spillovers  ”  arbitrarily  small.  The  authors 
then  state  that  the  PSW  theory  “  gives  some  information  on 
interchannel  and  intersymbol  interference  ....  However, 
the  relation  between  time  limiting  and  bandlimiting  .  .  .  need 
to  be  better  understood  ;  while  our  general  results  apply, 
the  identity  of  the  optimal  function  >/'  „  is  not  known  in  the 
case  that  B  is  a  projection  of  the  transform  into  .  .  .  passband.” 

The  third  part  of  the  publications  on  the  theory  of  PSW 
(Landau  and  Poliak  62)  arrived  at  the  results  on  the  TB 
product  versus  the  size  of  a  set  of  orthogonal  waveforms, 
which  we  discussed  in  section  M4. 

The  fourth  part  of  the  paper  (Slepian  64)  generalized 
the  work  of  the  previous  parts  to  functions  of  many  variables. 
Slepian  (65b)  derived  some  asymptotic  expansions  for 
PSW,  and  Slepian  and  Sohnenbliek  (65)  published  tables 
of  the  eigenvalues  associated  with  PSW  of  zero  order. 
Saltzberg  and  Ivurz  (64)  expressed  signals,  in  terms  of  PSW, 
which  were  derived  under  the  restraint  of  partial  and  total 
bandwidth  limitations.  They  show  how  their  signal  syn¬ 
thesis  method  can  be  used  successfully  to  improve  the  per¬ 
formance  of  feedback  communication  links.  Slepian  and 
Poliak  (61)  and  Landau  and  Poliak  (61)  stressed  the  im¬ 
portance  of  PSW  in  antenna  synthesis  problems  and  a 
correspondence  by  Kritikos  and  Dresp  (64)  gave  further 
results  in  this  direction.  Tung  (63)  in  a  correspondence 
note  related  the  new  PSW  theory  to  tho  sampling  theory 
based  on  sin  x  over  x  waveforms  ;  Rosenstark  and  Ivurz  (66) 
presented  a  conceptual  design  of  time  limited  signals  which 
are  optimized  against  coloured  Gaussian  noise  with  the  help 
of  PSW  theory.  Landgrebe  and  Cooper  (62  and  63)  used 
the  PSW  theory  for  two-dimensional  signal  representations. 

Tho  practical  application  of  orthogonal  sets  of  waveforms 
derived  with  the  help  of  the  PSW  t  heory  is  still  in  its  infancy, 
although  all  research  workers  seem  to  be  unanimous  in 
their  praise  of  the  fundamental  importance  of  the  new  theory 
for  signal  analysis  and  signal  synthesis.  Simon  and  Ivurz  (68) 
used  the  PSW  theory  for  the  frequency  domain  design  of 
signals  imbedded  in  Gaussian  noise.  Yet  there  are  some 
hopeful  beginnings  on  the  practical  side.  Saltzberg  and 
Ivurz  (65)  reported  that  they  investigated  waveforms  con¬ 
sisting  of  the  sum  of  a  few  PSW  for  a  high  bit  density  binary 
communications  system  requiring  strictly  bandlimited  signals. 
They  draw  positive  conclusions  about  tho  potential  impor¬ 
tance  of  such  signals  but  they  do  not  offer  any  suggestions 
for  their  generation.  Pettit  (65)  used  the  PSW  as  a  very 
efficient  orthogonal  set  in  a  comparison  with  six  other 
waveforms  such  as  rectangular,  triangular,  trapezoidal, 
exponential,  Gaussian,  and  cosine-squared.  Again  there  is 
no  report  in  that  paper  of  an  experimental  evaluation  of 
PSW  waveforms. 

A  still  broader  outlook  into  the  theory  of  wavefunctions 
is  due  to  Rhodes  (64),  who  investigated  some  problems  of  the 
double  orthogonality  properties  of  spheroidal  and  Mathicu 
functions. 

Concluding  the  subsection  on  perfectly  bandlimited  wave 
forms,  wo  recognize  that  the  PSW  had  a  significant  impact  in 
signal  theory.  There  is  hope  that,  in  the  coming  age  of 
computer-synthesized  waveform  alphabets,  the  practical 
importance  of  PSW  will  increase. 
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N5.  Other  Sets  of  Waveforms 

Although  the  sets  of  waveforms  created  direct  ly  by  filtering 
binary  sequences  and  modulating  them  on  a  carrier  are  the 
most  frequently  used  waveforms  today,  there  are  indications 
that  they  will  bo  replaced  in  the  future  by  bandlimited 
sets  of  waveforms  of  the  classes  discussed  in  this  section  and 
in  section  O.  The  crucial  problem  will  be  the  generation 
of  sufficiently  largo  sets  of  waveforms  with  a  large  TB 
product.  One  method,  which  is  frequently  suggested, 
will  be  the  synthesis  by  a  small  digital  computer.  Not 
much  has  been  published  about  the  operational  aspects  and 
the  cost  problems  of  such  procedures,  but  the  procedures 
have  been  applied  in  computer  simulation  in  studies  of 
various  classes  of  bandlimited  waveforms. 

Assuming  that  the  computer  synthesis  will  be  a  feasible 
method  for  the  generation  of  waveform  alphabets,  one  can 
consider  any  set  of  waveforms  of  arbitrary  complexity. 
The  problem  of  finding  analog  function  circuits  that  can 
produce  the  mathematical  function  of  the  waveforms  would 
no  longer  apply.  They  would  be  computed,  and  synthesized 
in  sampled  and  quantized  form.  Simple  smoothing  filters, 
together  with  analog  modulators,  would  finalize  the  waveform 
generation  in  the  carrier  band. 

Under  such  assumptions  it  is  interesting  to  notice  that 
Fermental  (65)  investigated  two  methods  of  synthesizing 
orthogonal  waveforms  with  all  members  of  a  set  having 
the  same  autocorrelation  function.  One  method  determines 
polynomials  of  differentiation  that  operate  on  a  pulse  to 
generate  the  waveforms.  The  other  method  makes  use  of 
trigonometric  polynomials  and  is  thus  related  to  the  trigo¬ 
nometric  product  waveforms  to  be  discussed  in  section  O. 

Dubrov  (61)  investigated  sets  of  Bessel  functions  as  a 
waveform  library  and  compared  it  with  sets  of  Jacobi, 
Gegenbauer,  Chebyshev,  and  Legendre  polynomials.  Unfor¬ 
tunately,  only  a  summary  of  this  interesting  work  has  become 
available  in  the  U.S.A.  in  translated  form.  Levan  (66) 
employed  the  theory  of  linear  operators  on  Hilbert  space 
for  a  class  of  linear  networks  to  show  that  it  is  possible  to 
construct  orthogonal  signals  in  the  time  domain  without 
using  the  conventional  Gram-Schmidt  procedure.  As  a 
special  case  this  approach  yields  the  exponential  waveform 
alphabets  discussed  in  subsection  Nil.  Hofstetter  (64) 
presented  a  constructive  procedure  for  determining  all 
possible  time-limited  functions  having  the  same  autocorrela¬ 
tion  function  as  a  given  time-limited  function.  Specific 
examples,  including  differentiated  rectangular  pulses,  para¬ 
bolic  ptdses,  and  other  pulsive  waveforms  with  discontinuities, 
show  how  to  apply  this  procedure.  Although  the  method  is 
applicable  to  exactly  time-limited  waveforms,  it  is  of  im¬ 
portance  to  bandlimited  waveforms,  too,  since  there  must 
always  be  both  a  band  limitation  and  a  time  limitation. 

The  above  few  examples  may  serve  as  an  indicat  ion  of  the 
wide  range  of  classes  of  functions  which  ult  imately  may  come 
into  consideration  as  waveform  alphabets.  The  selection  is 
far  from  exhaustive. 

0.  Sets  of  Product  Waveforms 

It  is  astounding  to  find  out  that  apparently  not  many 
attempts  have  been  made  to  define  waveform  alphabets  on  the 
basis  of  the  products  of  elementary  functions.  Yet  t  he  simple 
amplitude  modulation  process  is  based  on  just  such  a  mathe¬ 
matical  relationship.  It  is  easy  to  see  that  products  evidently 
can  define  zero  values  of  composite  functions  more  precisely 
than  sums  of  elementary  functions  can  do.  This  author, 
some  years  ago,  investigated  modulated  signals  and  discovered 
a  family  of  bandlimited  waveforms,  which  has  since  been 
extensively  explored.  These  waveforms  are  called  trigono¬ 
metric  product  waveforms  ( TPW ). 

Ol.  Definitions  for  Trigonometric  Product  Waveforms 

Trigonometric  product  waveforms  have  recently  been 
introduced  as  a  special  kind  of  bandlimited  signals  with  very 
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interesting  characteristics  (Filipowsky  67a  and  68b).  They 
are  defined  by  the  following  equation  : 

f„  *.  ,  (t)  =  A,  'n  sin  (is,  t  +  r,  ^  (109) 

s,  0  or  -(- 1  ;  r,  =  0  or  +  1  :  q  and  p  depend  on  t  he  choice 
of  s,  and  r,  . 

Trigonometric  product  waveforms  may  be  defined  on  the 
unbounded  interval  —  oo  <  t  <  +oo  or  on  the  bounded 
interval  [a,  b].  Particularly  useful  seems  the  bounded 
interval  —  jt/2  <  «,„  t  <  +  ir/2.  Using  that  interval  and 
substituting  x  <u,>  t  ;  A,  1,  one  can  see  that  a  typical 
trigonometric  product  waveform  may  have  the  following 
definition  : 

fi,  l2,  (x)  =  sin  x  .  cos  3x  .  sin  7x  .  cos  12x  (110) 

The  index  p  -  4,  indicates  the  order  of  the  waveform 
specifying  the  number  of  nontrivial  trigonometric  factors. 

The  index,  k  12,  represents  the  class  of  the  waveform, 
i.o.,  the  highest  integer  used  as  a  harmonic  factor  in  any  one 
of  the  trigonometric  factors.  It  is  a  useful  convention  to 
write  the  product  with  increasing  harmonics  going  from  left 
to  right.  Notice  that  the  definining  equation,  equation  109, 
permits  the  degeneration  of  any  trigonometric  factor  to  a 
factor  of  the  constant  value  “  1  ”.  This  happens  to  the  i-th 
factor  when  st  =  0  and  r,  -f  1  are  chosen.  In  the  example 
of  equation  110  this  particular  choice  has  been  made  for 
i  =  2,  4,  5,  6,  8,  9,  10,  11. 

The  index,  q  =  347,  is  called  the  rank  of  the  waveform. 
It  designates  the  place  of  any  particular  waveform  within 
an  ordered  list  of  all  possible  waveforms  of  a  given  class 
and  a  given  order.  The  example  in  equation  1  10  is  the  347th 
waveform  in  the  list  of  all  waveforms,  which  consist  of  four 
trigonometric  factors  (order  4)  and  which  use  the  twelfth 
harmonic  as  the  highest  harmonic  in  any  factor. 

The  method  of  ordering  all  these  waveforms  follows  a 
block  structure.  We  (’all  a  block  an  arrangement  of  2>‘  trigo¬ 
nometric  product  waveforms  where  all  waveforms  belong 
to  the  same  order  and  the  same  class,  and  all  have  exactly 
the  same  harmonic  factors.  The  waveform  of  equation  110 
belongs  to  the  block  : 

(1,3,7,12)  (111) 

The  ranking  within  a  block  follows  the  binary  counting 
method  but,  contrary  to  the  usual  practice,  the  lowest 
ranking  digit  is  on  the  left  side  and  the  highest  ranking 
digit  is  on  the  right  side.  The  sine  is  lower  ranking  than 
the  cosine.  The  first  waveform  (lowest  rank)  is  therefore 
the  waveform  with  all  sine  factors.  The  last  waveform 
(highest  rank)  is  the  waveform  with  all  cosine  factors. 
The  block  with  the  harmonic  factors  of  the  group  in  expression 
111  above  has  16  waveforms  that  must  be  listed  in  the 


following  order  : 

#1 

sin  x 

.  sin  3x  .  sin  7x  .  sin 

12x 

cos  X 

.  sin  3x  .  sin  7x  .  sin 

12x 

#3 

sin  x 

.  cos  3x  .  sin  7x  .  sin 

12x 

#4 

cos  X 

.  cos  3x  .  sin  7x  .  sin 

12x 

#3 

sin  x 

.  sin  3x  .  cos  7x  .  sin 

I2x 

#« 

cos  X 

.  sin  3x  .  cos  7x  .  sin 

12x 

#1 

sin  x 

.  cos  3x  .  cos  7x  .  sin 

12x 

#8 

cos  x 

.  cos  3x  .  cos  7x  .  sin 

I2x 

#9 

sin  x 

.  sin  3x  .  sin  7x  .  cos 

12x 

#10 

cos  x 

.  sin  3x  .  sin  7x  .  cos 

12x 

#H 

sin  x 

.  cos  3x  .  sin  7x  .  cos 

12x 
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4|=  12  :  cos  x  .  cos  3x  .  sin  7x  .  cos  12x 
=((=  13  :  sin  x  .  sin  3x  .  cos  7x  .  cos  12x 

4)=  14  :  cos  x  .  sin  3x  .  cos  7x  .  cos  12x 

=tj=  15  :  sin  x  .  cos  3x  .  cos  7x  .  cos  12x 

=jj:  16  :  cos  x  .  cos  3x  .  cos  7x  .  cos  12x 

The  total  number  of  blocks  in  order  p  and  class  k  can  be 
found  by  computing  the  total  number  of  different  arrange¬ 
ments  than  can  be  made  when  filling  k-1  positions  with  p-1 
numbers.  Notice  that  the  k-th  position  is  always  filled 
(with  the  number  k).  In  the  example  at  hand  there  are 
twelve  positions,  eleven  of  which  can  be  filled  with  three 
numbers.  Marking  the  unfilled  positions  (trivial  factors) 
with  a  dot,  one  arrives  at  the  following  first  arrangements. 


12  3 . 12 

12.4 . 12 

1.34 . 12 

.  2  3  4  .  12 

1  2  .  .  5 . 12 

1.3.5 . 12 

.23.5 . 12 

1  .  .  4  5 . 12 

.2.45 . 12 

..3  4  5  .  12 

1  2  ...  6 . 12 


etc. 

until  the  last  arrangement 
.  9  10  11  12 


The  total  number  of  blocks  is  : 


Nb 


(k  -  1)! 

(p-l)l(k-p)! 


(113) 


There  are  165  blocks  of  waveforms  of  order  4  and  class  12. 
Each  block  has  16  waveforms,  making  a  total  of  2640  wave¬ 
forms. 

It  is  easy  to  see  that  the  total  number  of  TPW  (the  highest 
rank  qm)  in  any  particular  subset  of  order  p  and  class  k  is 
therefore  : 


Qm 


2”  .  N B  =  2” 


(k  -  1)  ! 

(P  —  1)  !  (k  P)  ! 


(114) 


When  considering  that  0  !  =  1,  one  can  readily  compute  the 
following  Table  3  of  all  waveforms  of  all  classes  and  orders 
up  to  p  =  4,  k  =  4.  We  call  the  set  of  all  waveforms  of 
all  classes  and  all  orders  up  to  and  including  k  =  p  =  L, 
the  set  of  all  waveforms  of  level  L.  Table  3  lists  all  wave¬ 
forms  of  level  4.  S  stands  for  sino  ;  C,  for  cosine.  The 
column  in  which  S  or  C  is  listed  gives  the  harmonic  factor. 
The  waveform  f3.  ,.  3  (x),  listed  as  the  third  waveform  in 
the  group  1 1 ,  is  therefore  defined  as  : 

f3.  J>  3  (x)  =  sin  x  •  003  -x  •  s'n  4X 

Table  3  siiows  further  that  group  1 1  contains  all  waveforms 
of  order  3  and  class  4.  There  are  qm  =  24  waveforms, 
which  are  arranged  in  Nb  =  3  blocks  of  23  =8  waveforms 
each. 


Table  3 

All  trigonometric  product  wavoforms  and  blocks  at  level  4 


r 

“1 

Indices 

Harmon. 

Factors 

Indices 

Harmon. 

Factors 

P, 

k,  q 

i 

2 

3 

4 

P 

»k,q 

1 

2 

3 

4 

10 

CO 

II 

a 

=3; 

lm=® 

;  nb 

=1 

1 

p=0;  k 

-0;  fim-O;  nb=o 

3, 

3,1 

S 

s 

s 

0, 

3, 

3,2 

C 

s 

s 

0,0 

Trivial  Case 

3, 

3,  3 

s 

c 

s 

2 

p=l;  k 

=1;  q 

m=1 

nB 

=1 

3, 

3, 

3.4 

3.5 

c 

c 

s 

s 

s 

c 

o, 

1.1 

sinT 

=  1 

3, 

3,6 

c 

s 

c 

3, 

3,7 

s 

c 

c 

3 

p=i;  k 

=i;  c 

=2 

m 

;  nb=i 

3, 

3,8 

c 

c 

c 

1, 

1, 

1,1 

1  2 

s 

r 

P=3;  k 

=4; 

lm=24;  N 

B=3 

4,1 

3, 

s 

s 

s 

4 

p=1;  k 

=2;  q  =2 
m 

;  V1 

3, 

4,2 

C 

s 

s 

1. 

2,1 

■ 

s 

3, 

4,3 

s 

c 

s 

1, 

2,2 

c 

3, 

4,4 

c 

c 

s 

=1 

3, 

4,  5 

s 

s 

c 

b 

P=l;  k 

=3;  q 

nr2 

;  nb 

3, 

die 

c 

s 

c 

1, 

3,1 

s 

3, 

4,7 

s 

c 

c 

1, 

3,2 

c 

_3J_4J3_ 

_c_ 

c 

_c_ 

— 

— 

— 

— 

3, 

4,9 

s-1 

s 

s 

6 

p=i;  k 

=4;  9m=2 

;  nb=i 

3, 

4,  10 

c 

s 

s 

1, 

4, 1 

s 

3, 

4,11 

s 

c 

s 

1, 

4,2 

c 

3, 

4,12 

f 1 

o 

c 

s 

3, 

4,  13 

s 

s 

c 

7 

P-2;  k 

=2; 

1;  NB=1 

3, 

4,  14 

c 

s 

c 

2, 

2,1 

S 

s 

3, 

4, 15 

s 

c 

c 

2, 

2,2 

C 

s 

3, 

4,  16 

c 

c 

c 

2, 

2,3 

s 

c 

3, 

4,17 

s 

s 

s 

2, 

2,4 

c 

c 

3, 

4, 18 

c 

s 

s 

3, 

4,  19 

s 

c 

s 

8 

p=2;  k 

=3; 

lm=t 

;  NB=2 

3, 

4,20 

c 

c 

s 

2. 

3,1 

s 

s 

3, 

4,21 

s 

s 

c 

2, 

3,2 

c 

s 

3, 

4,22 

c 

s 

c 

2, 

3,3 

s 

c 

3, 

4,23 

s 

c 

c 

2, 

3,4 

c 

c 

3, 

4,  24 

c 

c 

c 

2 

3  5 

s 

s 

2, 

3,6 

C 

s 

12 

p=4;  k 

"4;  qm=16;  NB=1 

2, 

3,7 

s 

c 

4,4,1 

s 

s 

s 

s 

2, 

3,8 

c 

c 

4, 

4,2 

c 

s 

s 

s 

4, 

4,3 

s 

c 

s 

s 

9 

p=2;  k 

=4; 

lm=12;  NB=3 

4, 

4,4 

c 

c 

s 

s 

2, 

4,1 

S 

s 

4, 

4,5 

s 

s 

c 

s 

2, 

4,2 

C 

s 

4, 

4,6 

c 

s 

c 

s 

2, 

4,3 

S 

c 

4, 

4,7 

s 

c 

c 

s 

.2, 

4,4 

C 

c 

4, 

4,8 

c 

c 

c 

s 

2, 

4,5 

s 

s 

4, 

4,9 

s 

s 

s 

c 

2, 

4,6 

c 

s 

4, 

4,  10 

c 

s 

s 

c 

2 

4,7 

s 

c 

4, 

4,11 

s 

c 

s 

c 

2 

4,8 

c 

c 

4, 

4,12 

c 

c 

s 

c 

?- 

4,9 

s 

s 

4, 

4,  13 

s 

s 

c 

c 

2 

4, 10 

c 

s 

4, 

4, 14 

c 

s 

c 

c 

2 

4,11 

s 

c 

4, 

4,  15 

s 

c 

c 

c 

2 

4,12 

_ 

c 

c 

4, 

4, 16 

c 

c 

c 

c 
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For  computer  evaluation  it  is  important  to  identify  each 
waveform  by  its  order,  class,  and  rank.  While  order  and 
class  will  normally  bo  restricted  to  numbers  with  not  more 
than  two  decimal  digits,  one  can  see  that  the  rank  can  assume 
extremely  high  numbers.  The  number  of  all  TPWs  at  a 
certain  level  is  : 

N,rt  =  3t  (115) 

If  we  take  L  =  04  as  a  reasonable  upper  limit  for  practical 
systems,  we  recognize  that  wo  have  to  deal  with  a  fantastically 
largo  total  number  of  TPWs  : 

Nirz  =  3**  =  3.43302.  x  103°  (110) 

This  family  of  more  than  3.4  deeillions  of  different  wave 
forms  is  a  strong  asset  for  any  system  working  with  TPWs, 
but  it  forces  one  to  employ  a  reasonable  procedure  for  uni¬ 
quely  identifying  any  one  of  the  waveforms. 

To  identify  the  order,  class,  and  rank  of  any  desired 
waveform,  one  should  write  all  factors  of  the  desired  wave¬ 
form  in  the  following  component  notation  : 

(a!  Pi  i  a2  ^2  i  •  •  •  ai  Pi  !  •  *  •  0Cj>-1  Pd-  1  »  ap  Pp)  (HI) 

The  af  with  i  =  1,  2,  .  .  .  ,  p  are  the  harmonic  factors 
arranged  in  increasing  order  :  at  <  a,  <  .  .  .  ai  .  .  .  <  ap  . 
The  Pi  represent  the  sine  function  pt  =  S  or  the  cosine 
function  p t  =  C. 

As  an  example  we  shall  show  how  to  find  order,  class,  and 
rank  for  the  waveform  : 

f(x)  =  sin  2x  .  cos  4x  .  sin  5x  .  cos  12x  (118) 

This  TPW  is  symbolized  by  (2S  ;  4C  ;  5S  ;  12C).  (118) 

The  order  is  the  number  of  factors,  p  =  4  in  this  case. 
The  class  is  the  highest  harmonic  factor,  k  =  12  in  this  case. 
The  rank  can  be  calculated  with  the  help  of  the  following 
formula  (120)  when  placing  the  value  pp-m  =  0  for  pp-m  , 
m  =  0,  1,  ...  p  —  1,  if  the  Pi  term  in  1 17  is  S  ;  and  the  value 

PP-m  =  1,  if  the  Pt  term  in  117  is  C.  In  the  example  of 

equation  1 1 8  one  has  the  following  a  and  p  constants  : 

a,  =  2  ;  a2  =  4  ;  a3  =  5  ;  at  =  12 

^1  =  Pi  =  1  i  Pi  “  8  f  Pi  ”  1 

These  constants  together  with  p  =  4  and  k  =  12  have  to 
be  used  in  equation  120  to  get  the  rank  : 


q  =  2’’ 


y  1  ^  («p-,-<  —  2  —  j)  ! 

(p  -  2  -  i)  !  (a,-,.,  -  p  -  j  +  i)  ! 


+ 


+  ( 2 2  r~i-mp'-*') 

\  m-0  / 


+  i 


(120) 


In  the  special  example  of  expression  1 19  this  results  in 
the  following  numerical  values  : 

r  1  /1.2.3  1  .  2\  1/1.2  1\  1  /0!\q 

q  = 2<  in  (—  +  tt)  + 1  hr  +  0-)  +  o-  y  J 

+  8.  1+4. 0  +  2.  1  +  1.  0  +  1 

=  16  [4  +  3  +  1]  +  10  +  1  =  139 

When  applying  equation  120  it  is  necessary  to  recall  the 
following  identities  : 


0!  =  1  ; 


1  !  =  1  ; 


1 

(-")  ! 


=  0 


(121) 


The  expression  in  the  bracket  of  equation  120  gives  the 
number  of  waveforms  in  all  blocks  with  a  lower  rank  than 
the  block  in  which  the  desired  waveform  is  situated.  The 
rank  of  the  block  in  which  a  waveform  is  situated  is  therefore  : 


XT'*  1  V  -  2  -  j)  ! 

=  >  -  >  -  *  1  * - - - +  1  (122) 

ti  (P  -  2  -  i)  («_,.,  -  P  -  j  +  i)  ! 


For  example,  the  waveform  sin  2x  .  sin  3x  .  cos  4x  (2S  ; 
3S  ;  4C)  of  table  3  is  situated  in  order  p  =  3,  class  k  =  4, 
and  block  number  : 

r  =  1  r  <««  -  2> 1  i  .  1  r  («.  - 2) !  i  ,  j 

B  1  !  L(«2  -  3  -  0  +  0)  lj  0  !  L(a,  -  3  -  0  +  1)  lj 
=  1  +  1  +  1  =  3 

The  rank  of  the  selected  waveform  is  : 

q„  =  8  .  [2]  +  (2=  .  1  +  2 . 0  +  1  . 0)  +  1  =  21 
With  the  help  of  the  above  specifications  of  order,  class, 
rank,  and,  when  applicable,  of  level,  any  TPW  can  be  uniquely 
identified.  It  may  be  one  of  deeillions  of  different  wave¬ 
forms. 

02.  The  Signal  Base  of  Trigonometric  Product  Waveforms 
The  whole  Part  IV  of  this  paper  is  concerned  with  alphabets 
of  bandlimited  transmission  signals.  In  subsection  M4 
we  explained  the  fundamentals  of  bandlimited  waveforms 
and  we  showed  the  importance  of  the  TB  product  or,  rather, 
the  importance  of  the  signal  base  per  dimension  (pc). 

Before  concentrating  on  the  problem  of  the  signal  base 
of  TPW  sets,  it  may  be  advisable  to  show  a  number  of  typical 
TPWs  and  their  energy  density  spectra.  Figure  31  shows 
typical  waveform  pictures  for  a  number  of  examples  from 
first  order  to  fourth  order  and  from  first  class  to  eight  class. 


Figure  31. — Typical  time  functions  of  trigonometric  product  f  unctions 
of  various  orders  and  classes. 
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The  first  tkvo  waveforms  cos  x  and  sin  4  x  are  both  in  the 
first  order.  Notice  that  a  waveform  of  first  order  and  first 
class  comprises  only  one  half  oscillation  of  the  fundamental 
sine  or  cosine  wave.  Thus  we  recognize  that  the  trigono¬ 
metric  waveform  alphabets  discussed  hi  subsection  N3 
are  only  a  subset  of  all  TPWs  of  order  one.  In  particular 
they  are  the  subset  consisting  of  all  waveforms  contained 
I  in  the  set  cjf  TPWs  of  order  one  and  of  all  even  classes.  The 
second  example  in  fig.  31  (sin  4x)  is  one  waveform  of  this 
subset. 

Further  inspection  of  fig.  31  shows  that  all  the  TPWs 
in  this  example  are  either  even  or  odd  functions.  This  is  a 
fundamental  characteristic  of  all  TPWs.  Evidently,  it 
is  always  possible  to  find  a  quadrature  pair  of  TPWs,  where 
the  one  partner  of  the  pair  is  the  even  waveform  and  the  ot  her 
partner  is  Ithe  odd  waveform,  permitting  any  linear  com¬ 
bination  of  the  two  components  to  synthesize  any  unsym- 
metrical  waveform  of  that  special  variety.  Figure  31 
i  shows  that  any  waveform  with  an  odd  number  of  sine 
factors  results  in  an  odd  TPW.  Any  waveform  with  an 
even  number  of  sine  factors  results  in  an  even  TPW,  no 
matter  what  harmonic  factors  are  involved  in  the  individual 
factors.  This  is  easy  to  prove  when  considering  that  all 
sine  factors :  are  zero  for  x  =  0  but  have  a  positive  derivative 
at  x  =0,  while  all  cosine  factors  are  +1  at  x  =  0  with  a 
zero  derivative. 

Figure  31  shows  another  interesting  characteristic  of 
TPWs.  Waveforms  with  all  cosine  terms  form  a  positive 
pidse  at  x  0,  while  waveforms  with  all  sine  terms  have 
an  extended  valley  of  f(x)  =  0  near  x  =  0.  Figure  31  also 
|  shows  that!  only  one  of  the  eight  waveforms  selected  as 


typical  examples  has  a  non-zero  value  at  the  boundaries 
of  the  definition  interval.  Indeed,  it  is  easy  to  prove  that 
only  waveforms  with  all  sino  terms  exclusively  of  odd 
harmonics  and  with  all  cosine  terms  exclusively  of  even 
harmonics  can  produce  a  non-zero  value  at  the  limits  of  the 
interval.  Whenever  this  is  the  case,  the  non-zero  value 
must  be  the  peak,  value. 

Though  many  of  such  fundamental  characteristics  of 
TPWs  are  evident  from  an  inspection  of  waveform  pictures, 
it  is  deplorable  that  no  all-inclusive  theory  of  TPWs  has 
yet  been  developed.  We  are  still  in  a  kind  of  ma  thematically 
experimental  evaluation  of  this  new  family  of  waveforms. 

Figure  32  demonstrates  the  next  logical  step  in  the  explora¬ 
tion  of  TPWs  ;  the  computation  of  their  energy  density 
spectra.  Again  we  have  assembled  a  rather  arbitrary 
selection  of  TPWs  from  order  1  to  6  and  from  class  3  to  10. 
The  energy  density  spectra  are  plotted  to  the  right  of  each 
waveform  on  a  normalized  scale  over  q  =  f/f0  with  f„  = 
x/2vt.  This  means  that  q  =  1  corresponds  to  the  funda¬ 
mental  frequency  and  that  each  waveform  is  defined  over 
an  interval  from  —90°  (—  n/2)  to  +90°  ( -f  w/2)  of  the  angle 
of  this  fundamental  frequency  (f0).  The  first  waveform 
in  fig.  32  (sin  6x)  shows  this  very  clearly  by  displaying  a 
very  definite  maximum  of  energy  density  at  q  =  6. 

Comparing  all  energy  density  spectra  of  fig.  32b  with  the 
corresponding  waveform  pictures  of  fig.  32a  reveals  clearly 
the  fact  that  all  waveforms  are  essentially  bandlimited  to  a 
normalized  frequency 

P 

=  2  +  .  <123> 
(-i 


Figure  32. — Comparison  of  time  functions  ami  energy  density 
spectra  of  a  selection  of  TPWs. 
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where  the  a*  are  the  harmonic  factors  participating  in  the 
definition  of  any  waveform.  Take,  for  example,  the  first 
waveform  with  a,  =6  (t)l  =  8)  or  the  second  waveform  with 
«,  =  1,  a2  =  5  ;  (ijt  =  8),  or  the  last  waveform  in  fig.  32a 

with  ax  =  1  ;  a,  =  2  ;  a3  =  3  ;  a,  =  4  ;  a5  =  5  ;  a6  =  6 

(?;;,  =  23).  In  each  case  we  recognize  a  zero  of  the  spectral 
density  at  ij  l  and  we  see  that,  for  -q  —  -q  r,  ,  a  zero  occurs 

at  each  value  for  7;  =  gi,  +  2n  with  n  =  0,  1,  2 .  This 

means  that  the  essential  energy  of  any  waveform  stays 
within  a  frequency  band  from  zero  to  y  l  (in  normalized 
frequencies)  and  that  the  energy  density  for  frequencies 
t)  >  ijr,  is  relatively  small.  Evidently,  waveforms  with 
discontinuities  at  the  end  of  the  time  interval  show  a  larger 
percentage  of  energy  at  frequencies  r\  >  17  /,  .  Yet  even 

these  waveforms  follow  the  rules  established  above. 

This  leaves  us  with  the  problem  of  exactly  defining  the 
TB  product  (/3-value)  for  any  given  waveform  and  for 
any  exactly  defined  percentage  of  waveform  energy  outside 
the  TB  product.  Figure  33  shows  how  this  can  be  done 
for  various  TB  values.  As  TPWs  are  exactly  defined  in  the 
time  domain,  the  truncation  here  is  made  only  in  the  frequency 
domain.  A  waveform  truncated  in  the  frequency  domain 
will  produce  “  tails  ”  outside  the  definition  interval  in  the 
time  domain.  No  detailed  investigation  of  this  inter¬ 
dependence  between  frequency  and  time  domain  has  been 
made  yet  for  TPWs,  although  it  is  recognized  that  the  results 


waveforms,  or  whole  orders  and/or  classes.  The  precision 
of  plotting  can  be  adjusted  from  a  highest  precision  from  a 
minute  to  minute  to  a  quick-look  plot  from  degree  to  degree. 
When  plotting  the  waveforms  from  —90°  (—  7r/2)  to  +90° 
( -i-  7t/2)  with  the  finest  plotting,  the  printout  requires  180 
sheets  per  waveform  and  with  the  coarsest  plotting  it  needs 
only  three  sheets.  The  exact  values  of  the  waveform  for 
each  time  increment  can  be  printed  out  in  six  to  eighteen 
digit  numbers.  The  programs  have  been  written  for  IBM 
3G0/75  computer. 

In  addition  to  the  program  for  waveform  plotting,  programs 
for  the  calculation  of  amplitude  spectra  and  of  energy  density 
spectra  are  available.  No  systematic  evaluation  of  the 
signal  base  for  other  than  the  classical  orthogonal  sets  (to 
bo  discussed  next)  has  been  made  up  to  this  time. 

03.  Orthogonal  Sets  of  Trigonometric  Product  Waveforms 

Evidently  the  first  goal  of  the  investigation  of  trigono¬ 
metric  product  waveforms  was  the  search  for  orthogonal 
sets  of  bandlimited  waveforms.  Indeed  this  search  was  in 
the  background  of  the  author’s  mind  when  attempting  to 
create  better  bandlimited  waveforms  by  using  a  multiplica¬ 
tive  approach  to  avoid  discontinuities  in  the  waveforms. 

The  first  orthogonal  set  to  be  discovered,  and  for  some 
time  believed  to  be  the  only  orthogonal  set,  was  the  “  classical 
orthogonal  set  ”.  It  is  called  classical  because  it  has  a  most 


Figure  33. — Definition  of  the  TB  values  ( signal  bases)  for  a 
typical  TPW. 


with  PSWs  (subsection  N4)  will  be  indirectly  applicable 
for  any  such  study.  Figure  33  shows  that  this  particular 
waveform  (IS,  2S,  4S)  has  a  TB  product  of  4.00  when  about 
2.6  percent  of  Iho  energy  can  be  tolerated  outside  the  nor¬ 
malized  bandlimit  (ij  =  8.0;  TB  =  4.00).  This  sample 
waveform  is  one  of  a  set  of  eight  orthogonal  waveforms  which 
will  be  discussed  in  the  next  subsection. 

Computer  programs  are  available  to  plot  any  TPW  up 
to  rather  high  orders,  classes,  and  ranks.  The  computer 
can  be  instructed  to  plot  individual  waveforms,  blocks  of 
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regular  structure  by  making  each  harmonic  factor  at  a 
power  of  two  and  by  arranging  all  these  powers  of  two 
in  ascending  order.  A  classical  orthogonal  block  therefore 
follows  the  expression  : 

(1,  2,  4,  8,  ...  ,  2’-1)  (124) 

Figure  34  (Filipowsky  67a)  shows  the  eight  waveforms 
and  their  energy  density  spectra  of  the  classical  orthogonal 
set  of  order  3.  One  can  see  that  only  one  out  of  the  2 p 
waveforms  of  a  classical  orthogonal  set  has  a  discontinuity 
at  the  end  of  the  interval.  It  has  been  stressed  in  subsection 
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Figure  34. — The  time  functions  rind  the  spectra  of  the  classical 
orthogonal  set  of  TFWs  of  order  3. 


02  that  this  criterion  for  a  discontinuity  applies  quite 
generally  o  all  TPWs,  not  only  to  orthogonal  sets.  Indeed, 
it  can  be  shown  that  only  one  combination  of  sines  and  cosines 
in  each  block  of  waveforms  can  fulfill  this  criterion.  To 
find  in  any  block  this  one  waveform  with  the  discontinuities, 
one  must  use  the  sine  function  for  all  odd  harmonics  and  the 
cosine  function  for  all  even  harmonics.  Therefore,  in  all 
classical  orthogonal  sets,  the  waveform  with  the  structure 
(IS,  2C,  40,  8C . 2>‘~'C)  must  bo  the  one  with  the  dis¬ 

continuity!  According  to  our  ranking  principle  this  is 
always  the  next  to  last  waveform  in  a  classical  orthogonal 
set.  This  singular  occurrence  of  a  wavoform  with  discon¬ 
tinuities  is  one  of  the  most  attractive  features  of  TPWs. 
The  larger  the  order  of  the  waveform,  the  smaller  is  the 
percentage  of  waveforms  with  discontinuities.  Evidently, 
the  TPWs  of  the  first  order  have  50  percent  of  all  waveforms 
with  discontinuities.  This  is  still  the  case  when  only  classes 
of  even  harmonics  are  used  for  first  order  waveforms,  as 
proposed  for  the  trigonometric  sets  of  section  N3.  One 
may  get  a  subset  of  first  order  waveforms  that  is  free  of 
discontinuities  by  using  only  sine  functions  for  the  wave¬ 
forms  with  an  even  harmonic  factor  and  only  cosine  functions 
for  the  waveforms  with  an  odd  harmonic  factor,  thus  omitting 
50  percent  of  all  first  order  TPWs.  Still  even  such  subsets 
will  have  ja  strong  spill-over  in  the  frequency  domain  due 
to  the  discontinuities  in  all  derivatives  of  such  subsets. 
TPWs  of  higher  order  are  definitely  preferable. 

To  avoid  any  harmful  effect  of  that  one  waveform  with  a 
discontinuity  in  any  block  of  waveforms  of  higher  order, 
orthogonal  or  nonorthogonal,  the  suggestion  has  been  made 
(Filipowskv  08b)  to  uso  this  waveform  for  synchronization 
purposes.  If  it  is  used  with  alternate  polarity  in  alternate 
intervals  and  if  it  is  used  with  constant  amplitudes,  the 
spectrum  of  this  waveform  degenerates  to  a  line  spectrum 
with  the  highest  frequency  component  nt  tj  =  7  (in  the 
example  of  fig.  34).  If  the  set  is  an  orthogonal  set  and  if 


exactly  clocked  resampling  is  applied,  the  synchronization 
waveform  can  be  applied  continuously  without  interfering 
with  the  other  waveforms  that  may  be  used  for  message 
transmission. 

There  is  one  other  characteristic  waveform  in  fig.  34  and 
that  is  the  last  one  of  the  set.  It  has  cosine  factors  only 
and  it  resembles  a  bell-shaped  pulse  or  a  sine  x  over  x  pulse 
at  the  centre  of  each  transmission  interval.  The  energy 
density  spectrum  of  such  a  pulse  has  a  nearly  uniform  dis¬ 
tribution  over  the  available  frequency  band.  Accordingly, 
the  suggestion  has  been  made  (Filipowsky  68b)  to  avoid 
using  this  waveform  for  message  transmission  but  rather 
use  it  in  the  receiver  correlator  as  a  “  noise  window  This 
means  that  the  noisy  composite  signal  in  the  receiver  will 
be  correlated  with  this  waveform  as  it  will  be  correlated 
with  all  the  other  waveforms.  Since  this  waveform  never 
will  be  transmitted  (no  component  of  the  messago  vector 
will  be  in  the  dimension  of  this  waveform),  the  correlator 
output  corresponding  to  this  waveform  will  be  duo  only  to 
external  disturbances  (noise).  Thus  the  receiver  can  “  look- 
through  ”  the  message  signals  to  see  the  noise.  Because  of 
the  uniform  frequency  spectrum  of  this  “  window  waveform  ”, 
it  is  likely  that  any  random  noise  components  will  be  traced. 
Noise  components  in  the  exact  dimensions  of  the  other 
waveforms  cannot  be  noticed.  Obviously,  these  components 
are  the  ones  causing  errors  ;  evidently,  the  noise  window 
cannot  compute  the  exact  position  of  such  noise  vectors. 
Yet,  the  larger  the  dimensionality  of  the  set,  the  smaller 
will  bo  the  probability  that  a  random  noise  vector  will  have 
zero  component  value  in  the  dimension  of  the  noise  window 
and  high  component  values  in  any  other  dimension.  Indeed, 
for  many  practical  systems,  small  band  disturbances  (cross 
talk  from  other  carriers,  CW  jamming,  etc.)  will  be  the 
principal  noise  source.  Any  such  disturbance  must  have  a 
significant  component  in  the  dimension  of  the  noise  window. 
Averaging  the  noise  window  output  over  many  intervals 
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can  give  a  good  indication  of  any  quasistationary  random 
disturbance.  Evasive  measures  can  then  be  taken  in  any 
adaptive  communications  system. 

Tables  4  and  5  give  the  TB  products  of  t.ho  classical 
orthogonal  sets  of  order  and  order  4  respectively.  In 
these  tables  the  waveforms  are  listed  in  the  order  of  their 
signal  base  (TB),  with  the  waveform  with  the  smallest 
signal  base  coming  first.  The  waveform  with  the  discon¬ 
tinuity  has  the  largest  signal  base  and  therefore  comes  last 
in  the  tables.  It  can  be  estimated  from  the  values  of  table  5 
that  15  waveforms,  when  permitting  12  percent  of  their 
energy  outside  the  TB  value,  would  have  an  average  signal 

Table  4 

TB  products  for  the  classical  orthogonal  set  of  TPWs  of  order  3 


Typical  waveform  equation:  sin  x,  sin  2x 

sin  4x 

Number 

%  of  waveform  energy  outside  TB 

2% 

4% 

6% 

oCt 

O  0 

10% 

15% 

f3,4,2 

3.85 

3.  68 

3.57 

3.50 

3.43 

3.28 

f3,4,6 

3.85 

3.68 

3.57 

3.50 

3.43 

3.28 

f3,4,4 

3.95 

3.72 

3.60 

3.52 

3.45 

3.  34 

f3,4,8 

3.95 

3.72 

3.60 

3.  52 

3.45 

3.  34 

f3,4,  1 

4. 1G 

3.86 

3.82 

3.74 

3.64 

3.42 

f3,4,  5 

4. 18 

3.92 

3.86 

3.78 

3.  70 

3.48 

*3,4,3 

4.82 

4.08 

3.96 

3.88 

3.78 

3.62 

f3, 4,  7 

8.40 

7.40 

6.80 

6.  70 

5.80 

5.20 

base  of  about  7.04.  This  means  that  the  number  of  wave¬ 
forms  in  this  orthogonal  set  (15)  is  about  2.13  times  the  signal 
base  ;  i.o.,  the  theoretical  value  of  2.4  times  the  signal  base 
is  being  approached.  No  computations  have  been  mado 
for  tbe  signal  bases  of  sets  of  higher  order  (beyond  p  =  4), 
but  a  comparison  of  tablo  4  with  table  3  does  permit  the 
expectation  that  sets  of  seventh  or  eighth  order  (128  or 
250  waveforms)  will  come  very  close  to  the  theoretical 
minimum  signal  base.  More  important  may  be  the  perfor¬ 
mance  under  extremely  tight  band-limitation.  Table  5  shows 
that  12  of  the  10  waveforms  have  a  signal  base  of  10.25  or  less 
when  only  0.1  percent  of  the  energy  can  be  permitted  outside 
the  band  and  time  limits.  It.  will  bo  interesting  to  see  how 
these  figures  improve  still  further  when  going  to  sets  of  still 
higher  order. 

This  author  was  rather  surprised  to  discover  that  there 
exist  many  more  sets  of  orthogonal  TPW  than  merely  the 
classical  orthogonal  sets.  Yet  the  indications  are  that  the 
classical  sets  are  the  best  bandlimited  sets.  Naturally, 
it  was  important  to  find  an  algorithm  that  can  indicate 
from  the  harmonic  factors  of  a  block  if  this  block  will  repre¬ 
sent  an  orthogonal  set  or  not.  Such  an  orthogonal  algorithm 
has  recently  been  found.  One  simply  inspects  all  partial 
sums  that  can  bo  formed  from  all  the  harmonic  factors  of  a 
block.  If  any  two  partial  sums  (which  must  not  contain 
the  same  harmonic  factor  twice)  aro  equal,  the  block  is 
nonorthogonal.  If  no  such  sums  aro  equal,  the  block  is 
orthogonal.  Single  harmonic  factors  count  as  a  partial 
sum.  A  few  examples  will  show  this  procedure.  Wo 
may  take  some  of  the  blocks  of  order  4  and  class  12  as  they 
aro  listed  in  subsection  Ol  : 


(1,  2,  3,  12) 
(1,2,  4,  12) 
(I.  3,  4,  12) 
(1,3,  7,  12) 
(3,  4,  5,  12) 


nonorthogonal 

orthogonal 

nonorthogonal 

orthogonal 

nonorthogonal 


1+2=3 

1+3=4 

3  +  4  +  5  =  12 


Tablo  5 

TB  products  for  the  classical  orthogonal  set  of  TPWs  of  order  4 


I# 

Harmonics 

Percentage  fi  c(  energy 

outside  T  6 

1 

2 

3 

4 

0.1T 

0.  5% 

1% 

2T 

3  0 

5*o 

10% 

'4,8,22 

C 

s 

c 

c 

6. 17 

7.9G 

7.85 

7.69 

.7.61 

7.45 

7. 19 

'<,8,30 

C 

s 

c 

c 

8. 10 

7.98 

7.85 

7.67 

7.60 

7.47 

7.20 

'4,8,18 

c 

s 

s 

s 

8. 18 

7.92 

7.83 

7.76 

7.58 

7.42 

7.  18 

*4,8,20 

c 

s 

s 

c 

8. 18 

7.96 

7.83 

7.71 

7.60 

7.42 

7. 19 

*4,8,20 

c 

c 

s 

s 

8.32 

8.03 

7.91 

7.74 

7.63 

7.49 

7.  18 

*4,8,28 

c 

c 

s 

c 

8.32 

8.03 

7.91 

7.74 

7.69 

7.30 

7.  18 

*4,8,24 

c 

c 

c 

s 

8.50 

8.08 

7.95 

7.76 

7.67 

7.50 

7.  18 

'4,8,32 

c 

c 

c 

c 

8.82 

8.  10 

7.97 

7.81 

7.68 

7.51 

7.21 

*4,8,17 

s 

s 

s 

s 

9.00 

8.  10 

8.00 

7.8G 

7.80 

7.61 

6.78 

*4,8,25 

s 

s 

s 

c 

9.04 

8.  18 

8.04 

7.89 

7.77 

7.63 

6.74 

*4,8,21 

s 

s 

c 

s 

9.25 

8.22 

8.  11 

7.93 

7.84 

7.66 

7.  10 

*4,8,29 

s 

s 

c 

c 

10.25 

8.50 

8.  15 

7.97 

7.86 

7.70 

7.23 

*4,8,19 

s 

c 

s 

s 

10.91 

9.04 

8.31 

8.05 

7.96 

7.81 

7.42 

*4,8,27 

s 

c 

s 

c 

14.00 

10  00 

9.00 

8. 19 

8.08 

7.91 

7.70 

*4,8,23 

s 

c 

c 

s 

18.  50 

12.52 

10.75 

9.21 

8.81 

0.  12 

7.78 

*4,8,31 

s 

c 

c 

c 

29.50 

25.00 

22.50 

19.00 

17.20 

14.52 

11.  10 

(0,  10,  11,  12)  nonorthogonal  9  +  12  =  10  +  11 

Combining  this  investigation  with  equation  123  for  band- 
limitation  indicates  that  the  elasical  orthogonal  set  is  usually 
the  set  of  the  lowest  class  in  a  given  order.  As  the  class  is 
increased,  more  and  more  orthogonal  sets  can  be  found  in 
the  same  ordor,  until  the  class  is  high  enough  to  yield  a 
classical  orthogonal  set  of  the  next  higher  order.  To  demon¬ 
strate  this  behaviour,  wo  start  with  ordor  4.  The  classical 
set  is  (1,  2,  4,  8)  with  v  l  =  17;  there  aro  8  other  orthogonal 
sets  in  this  order  and  class.  They  are  tabulated  below 
together  with  vr  ,  their  normalized  bandlimitation  (see 
equation  123). 


(2,  3,  4,  8) 

VL  =  19 

(4,  5,  0,  8) 

VL  =  25 

(2,  4,  5,  8) 

TJ  L  =  21 

(2,  4,  7,  8) 

<3 

II 

to 

CO 

(1,  4,  6,  8) 

TIL  =  21 

(3,  6,  7,  8) 

VL  =  26 

(3,  4,  6,  8) 

t]l  —  23 

(4,  6,  7,  8) 

VL  =  27 

However,  there  is  one  orthogonal  set  of  order  4  and  a  class 
lower  than  8.  This  is  (3,  5,  6,  7)  with  ij /,  =  23.  Continuing 
this  search  for  orthogonal  sets  in  order  4  one  will  find  eleven 
orthogonal  sots  for  class  9,  thirty-four  sets  for  class  10, 
forty-five  sets  for  class  II,  seventy-seven  orthogonal  sets  for 
class  12,  and  so  forth.  All  these  sets  of  classes  higher  than 
2c- 1  require  wider  bandwidth  than  the  classical  set,  which  is 
of  class  2P“*. 
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Other  interesting  properties  of  orthogonal  sets  and  also  of 
nonorthogonal  sets  are  still  under  investigation  at  the  time 
of  this  writing.  Autocorrelation  programs  and  cross¬ 
correlation  programs  have  been  written  for  the  IBM  360/75 
computer.  Cross-correlation  matrices  can  be  easily  plotted 
and  noise  (  in  the  form  of  random  samples  of  a  given  distribu¬ 
tion)  can  be  added.  Preliminary  results  of  this  continuing 
investigation  seem  to  indicate  that  at  least  the  classical 
orthogonal  sets  seem  to  be  autocorrelation  invariant.  Many 
similar  conjectures  can  be  verified  experimentally  on  the 
computer,  but  rigorous  mathematical  proofs  for  many 
of  the  outstanding  characteristics  of  TPW  are  still  missing. 

04.  Future  Possibilities  for  Trigonometric  Product  Waveforms 

Much  research  is  still  required  before  claims  can  be  made 
that  we  really  know  all  the  characteristics  of  TPWs.  First 
of  all,  it  will  be  necessary  to  establish  the  connection  to  the 
still  wider  class  of  trigonometric  polynomials.  Evidently 
each  TPW  of  order  p  can  be  expanded  into  a  trigonometric 
polynomial  of  Up-'  terms.  Second,  it  will  be  beneficial  to 
investigate  TPWs  in  the  complex  domain  and  establish 
their  relationship  to  orthogonal  exponentials.  Third,  there 
is  not  only  the  large  family  of  nonorthogonal  blocks,  but  also 
the  cross -correlation  between  different  orthogonal  sets  that 
await  further  investigation. 

There  is  no  doubt,  however,  that  the  characteristics  of 
TPWs  that  have  been  uncovered  so  far  will  assure  the  con¬ 
tinued  interest  in  these  waveforms.  One  may  remember 
that  sine  and  cosine  functions  are  the  backbone  of  all  com¬ 
munications  and  they  temselves  are  derived  from  the  wheel, 
the  greatest  invention  of  the  human  race. 

Conclusions 

This  comprehensive  review  of  the  fundamentals  of  non¬ 
binary  information  transmission  has  shown  : 


1.  Nonbinary  systems  are  theoretically  superior  to  binary 
systems. 

2.  Multidimensional  nonbinary  systems  are  essential  for 
significant  progress  in  this  area. 

3.  Multidimensional  codebooks  have  a  long  history  of 
research  and  many  of  the  results  of  error-correcting 
codes  can  be  applied  to  systems  with  message  decoding. 

4.  The  research  for  bandiimited  waveforms  has  lagged 
behind  the  codebook  research  but  it  is  now  catching  up. 

5.  Only  a  few  practical  nonbinary  systems  are  in  operation 
at  this  time,  but  many  experimental  systems  are 
under  investigation. 

6.  When  attempting  to  close  the  margin  of  20  to  30  decibels 
of  utility  that  still  separates  the  experimental  systems 
from  their  mathematical  models,  designers  will  require 
large  scale  integration  technology  and  computer-like 
system  organization. 
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APPENDIX  A 


Appendb  A 

List  of  Abbreviations 


A/D 

ADEM 

AFC 

AFSK 

AGO 


AM 

APR 

ARQ 

ASK 

BC 

BCH 

BDD 

BFC 

bps 

BTL 

CCTR 

C\V 

db 

dc 


Analog-to-digital  (conversion) 

Adaptively  data  equalized  modem 
Automatic  frequency  control  (modem) 

Audio  frequency  shift  keying 
Automatic  gain  control  (=  AVC,  automatic 
volume  control) 

Amplitude  modulation 
Automatic  phase  reference  (class) 

Automatic  request  for  repetition  (Morso  code 
symbol) 

Amplitude  shift  keying  (on-off  keying) 
Bose-Chaudhuri  (code) 
Bose-Chaudhuri-Hoghem  (code) 

Bounded  distance  decoding 

Binary  fading  channel 

Bits  per  second 

Bell  Telephone  Laboratories 

Carrier  coherent  telemetry  rate  (subsystem) 

Continuous  wave  (unmodulated  carrier) 

Decibel  (logarithmic  unit) 

Direct  current  (constant  value  component) 


DC-MPSK 

DCPSK 

DDT 

DELTIC 

DNRZ 

DSB-SC 


DSIF 

exp 


FD 

FM 

F5LDFB 

FSK 

HF 

HOA 

IF 

ISIC 

JPL 

LES 

LET 

LSI 


Differentially  coherent-multiphase  shift  keying 
(system) 

Differentially  coherent  phase  shift  keying 
(system) 

Digital  data  transceiver 
Delay  line  time  compressor 
Differential  non-return  to  zero 
Double  sideband-suppressed  carrier  (modula¬ 
tion) 

Deep  spaeo  instrumentation  facility 
Mathematical  symbol  for  the  exponential 
function 

Frequency  division  (in  multiplex  systems) 
Frequency  modulation 

FM  demodulator  with  feedback  (aLso  F5IFB) 
Frequency  shift  keying 
High  frequency 

Higher  order  alphabet  (nonbinary) 
Intermediate  frequency  (also  i.f.) 

Intersymbol  interference  corrector 
Jet  Propulsion  Laboratory 
Lincoln  experimental  satellite 
Lincoln  experimental  terminal 
Large  scale  integration 
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MASK 

Mulfciainplitude  shift  keying  =  multilevel 
system 

QFT 

QPPM 

Quantized  frequency  transmission 

Quantized  pulse  position  modulation  (system) 

MDD 

Minimum  distance  decoding 

RC 

Resistor-capacitor  (circuits) 

MF 

Medium  frequencies  (300  kHz  to  3  MHz) 

RCA 

Radio  Corporation  of  America 

MFSK 

Multifrequency  shift  keying 

RM 

Reed-Muller  (codes) 

MIT 

Massachusetts  Institute  of  Technology 

RMS 

Root-mean -square  (value) 

MLD 

Maximum  likelihood  decoder  (or  decider) 

RPT 

Relative  phase  telegraphy 

MODEM 

Modulator-demodulator  (combination) 

RZ 

Return-to-zero 

MPSK 

Multiphase  shift  keying 

SD 

Synchronous  demodulator 

MTSK 

Multitime  shift  keying 

SEC'O 

Self-regulating  error  correcting  coder-decoder 

MUX 

Multiplex  (operation,  system,  equipment,  etc.) 

SNR 

Signal-to-noise  (ratio) 

NCFSK 

Non-coherenfc  frequency  shift  keying  (system) 

S/P 

Serial-to-parallel  (conversion) 

NRZ 

Non-return-to-zero 

SPR 

Separate  phase  reference  (class) 

PAM 

Pulse  amplitude  modulation 

SR 

Shift  register 

PAR 

Peak-power  to  average-power  ratio 

SSB 

Single  sideband  (modulation) 

PCM 

Pulse  code  modulation 

TASI 

Time  assignment  speech  interpolation  (system) 

PhM 

Phase  modulation  (analog) 

TB 

Time  bandwidth  (product) 

PLL 

Phase  lock  loop 

TPC 

Time  polarity  control 

PLLFB 

Phase  lock  loop  with  feedback 

TPW 

Trigonometric  product  waveforms 

PMCM 

Pulse  Morse  code  modulation 

USAF 

United  States  Air  Force 

PN 

Pseudo-noise 

vco 

Voltage  controlled  oscillator 

PR 

Pseudo-random 

V 1 F 

Vestigial  interpolation  filter 

P/S 

Parallel-to -serial  (converter) 

VOCODER 

Voice  coder  and  decoder 

PSI 

Pseudo-sample  insertion  (technique) 

VSB 

Vestigial  sideband  (modulation) 

PSK 

PST 

PSW 

Phase  shift  keying  (system) 

Paired  selective  ternary  (transmission) 

Prolate  spherical  wave  functions 

WJ 

Wozencraft  and  .Jacobs  (66),  (reference  to  their 
book,  “  Principles  of  Communications  En¬ 
gineering  ”) 

PWS 

Predicted  wave  signalling  (Kineplex) 

XMTR 

Transmitter  (also  Xmtr) 

APPENDIX  B 

LIST  OF  SYMBOLS 


Symbol 

Quantity 

Unit 

MKS 

Dimension 

A 

SNR  parameter  used  by 
Shannon  (59).  (A  =  V") 

Number 

m 

a<# 

Coordinates  in  orthonormal 
signal  space. 

Number 

in 

B 

Bandwidth  (in  transmission 
channels,  equal  to  B  lft 
unless  stated  ditTerently). 

Hertz 

[s-'l 

B ,// 

Effective  bandwidth 

Hertz 

Is-'] 

B,„ 

Effective  bandwidth  of  the. 
information-carrying  signals 
at  the  input  to  the  trans¬ 
mitter. 

llertz 

Is-'] 

B„, 

Effect  ive  bandwidth  of  the  in¬ 
formation-carrying  signals  at 
the  output  from  the  receiver. 

Hertz 

[s-'l 

BS 

Effective  bandwidth  of  one 
subchannel  in  a  MFSK  sys¬ 
tem. 

Hertz 

[»-■] 

B„ 

Effective  bandwidth  of  the 
transmission  channel  (if  spe¬ 
cifically  designated). 

Hertz 

[s-q 

c 

Shannon's  channel  capacity. 

Bit  per  second 

[11 

C SH 

Shannon's  channel  capacity 
in  nits  (natural  logarithmic 
units)  per  dimension  (degree 
of  freedom). 

Nit  per  dimen¬ 
sion 

[1] 

CO.  D) 

The  “  constant  ”  of  a  partic¬ 
ular  exponential  bound  of  the 
error  probability  of  coded 
mult  {dimensional  systems. 

Number 

in 

DB  or  I) 

i  191 

D, 

Bit  density  of  Shannon’s 
ideal  system. 

Number 

in 

Symbol 

Quantity 

Unit 

MKS 

Dimension 

db 

Symbol  for  decibel  indicating 
that  ten  times  logarithm  of 
base  ten  has  to  be  used  of  the 
magnitude  to  which  db  is 
placed  as  superscript. 

Symbol 

<1,7 

Distance  between  the  i-th 
and  j-th  code  word. 

BSI 

m 

E 

Average  signal  energy  (aver¬ 
age  over  all  members  of  a  set 
of  signals). 

[VAsJ 

Efl 

Average  signal  energy  per  bit 
of  output  information. 

Watt  second  per 
bit 

[VAs] 

E  D 

Average  signal  energy  of  one 
digit. 

Watt  second  per 
digit 

IV  As] 

E  1.0.  u> 

Reliability  function  of  an  ex¬ 
ponential  bound. 

Number 

[1] 

E, 

Energy  of  a  particular  signal 
f. . 

Watt  second 

IVAs) 

e 

Error  rat  io  :  the  bar  indicates 
that  error  counts  need  to  be 
averaged  over  long  intervals. 

HI 

«/* 

Bit  error  rat  io. 

HI 

ce 

Character  error  ratio. 

Number 

[11 

*D 

Digit  error  ratio. 

Number 

HI 

^EB 

Equivalent  bit  error  ratio. 

Number 

HI 

c  w 

Word  error  ratio  (average 
probability  of  a  wrong  deci¬ 
sion  when  receiving  one  word 
out  of  a  codebook  of  M 
words). 

Number 

[1] 
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Symbol 

Quantity 

Vnll. 

MKS 

Dimension 

r. 

Fundamental  frequency  of  a 
TPW. 

Hertz 

[H-’l 

K  no 

Boltzman’s  constant 
(1.38044.10-**) 

Watt  second  per 
degree  Kelvin 

|VAh  K  <] 

k 

Number  of  digits  in  a  sequence 
of  digits.  Also  :  class  of  a 
trigonometric  product  wave¬ 
form. 

Number 

[11 

L 

The  largest  possible  number 
of  orthonormal  functions 
that  can  exist  in  a  certain 
Til  product  under  specified 
restraints.  Also :  level  of 
trigonometric  product  wave¬ 
forms. 

Number 

11] 

L' 

A  numerical  magnitude  used 
in  Part  III. 

Number 

111 

l  or  1 

Number  of  elements  in  a 
sequence  of  elements  (in  a 
digit  or  in  a  word). 

Number 

III 

M 

Number  of  rows  (code  words) 
in  the  codebook  matrix  (num¬ 
ber  of  different  sentiences  of 
digits  available  for  trans¬ 
mission). 

Number 

III 

m 

Number  of  digits  (states  or 
levels  in  a  nonbinary  signal 
alphabet). 

Number 

ID 

m< 

Number  of  elements  in  the  i-th 
group  of  numbers  in  permu¬ 
tation  modulation. 

Number 

ID 

m# 

1  Number  of  dimensions  of  a 
multidimensional  signal. 

Number 

111 

me 

Number  of  characters  in  a 
nonbinary  output  alphabet. 

Number 

HI 

N 

Average  noise  power  in  a 
given  bandwidth. 

Watt 

[VA] 

N. 

Noise  power  density  of  white 

1  noise. 

Watt  second 

IVAf| 

XF 

Noise  figure  (in  decibels)  or 
noise  factor  (in  numbers). 

Number  or  deci¬ 
bel 

HI 

n 

Number  of  bits  carried  by  one 
digit  or  by  one  sequence  of 
digits  (see  R nh). 

Number 

in 

n(t) 

Noise  waveform,  random  func¬ 
tion. 

•Neutralized 

[V/*a'/*i 

P 

“  Power  ”  of  a  code  word 
(see  11k.  HI). 

Number 

HI 

Pc 

Probability  of  correct  deci¬ 
sion. 

Number 

111 

p 

Order  of  a  trigonometric  pro¬ 
duct  waveform. 

— 

q 

Number  of  waveforms  in  the 
waveform  library.  Also: 

rank  of  a  TPW. 

Number 

HI 

Qo 

Number  of  orthonormal  func¬ 
tions  in  a  set  of  such  func¬ 
tions. 

Number 

HI 

Q(x) 

A  special  form  of  the  prob¬ 
ability  integral  (see  fig.  8b). 

Number 

HI 

R 

Transmission  rate 

Bit  per  second 
or  other  units. 

Is-] 

R 

Average  transmission  rate 
(for  time- variable  systems). 

Bif  per  second 
or  other  units 

[s-l 

11 B 

Transmission  rate. 

Bif  per  second 

|H-‘I 

It. 

Binary  exponential  bound 
parameter  as  used  by  Wozen- 
craft  and  Jacobs  (65). 

Number 

HI 

Jto* 

Nonbinary  exponential  bound 
parameter  as  used  by  Wozen- 
craft  and  Jacobs  (65). 

Number 

III 

Ryn 

Nyquisf  and  Hartley’s  in¬ 
formation  content  of  a  non¬ 
binary  word  (also,  see  n). 

Bit  per  word 

ID 

Rsj/ 

Shannon’s  information  rate  in 
nits  per  dimension. 

Nits  per  dimen-  I 
sion 

HI 
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Symbol 

Quantity 

I'nit 

MKS 

Dimension 

r 

Radius  of  a  sphere  in  signal 

1  space. 

;  •'Neutralized 

S 

Average  signal  power  taken 
over  many  message  intervals 
(assuming  stationary  signal 
and  message  statistics). 

Watt 

IVAJ 

8 

Peak  signal  power  during  any 
signal. 

Watt 

[VA] 

s, 

Average  signal  power  of  the 
i-th  signal  waveform  of  an 
alphabet  of  q  waveforms  or 
of  the  i-th  message  element 
of  a  message  sequence  of 

1  elements 

Watt 

[VA] 

8|  (t) 

I-th  signal  function  (wave- 
!  form). 

•Neutralized 

l  V'CA1/1] 

Tb 

Bit  period,  the  reciprocal 
value  of  R. 

Second 

Is] 

Tf 

Duration  of  one  time  slot 
reserved  for  one  character  of 
the  waveform  library. 

Second 

w 

Tb 

Duration  of  one  digital  inter¬ 
val  (usually  identical  with 

T  c). 

Second 

Is] 

T.u 

Duration  of  one  message 
interval. 

Second 

[si 

T., 

Average  operational  noise 
temperature. 

Degree  Kelvin 

[°-K] 

Tir 

Duration  of  one  transmission 
word  consist  ing  ofl  time  slots 
of  length  Tc  each. 

Second 

[s] 

u 

Utility  (  bit  density  over 
power  contrast). 

Number  or  deci¬ 
bel 

HI 

Ui 

Utility  of  Shannon’s  ideal 
system. 

Number 

[1] 

w< 

Word  vector  in  /  dimensional 
codebook  space. 

Vector 

Hi 

W Lit 

Transmission  vector  (word) 
in  linear  real  coding  (LR). 

Vector 

HI 

a 

Input  vector  with  real  num¬ 
bers  a<  as  coe indents. 

Vector 

HI 

i-th  harmonic  factor  defining  a 
TPW. 

Number 

HI 

0 

Signal  base  (TB  product). 

X  timber 

111 

f*n 

Signal  base  for  one  bit  of 
t ransm i t ted  information. 

Number 

HI 

Pc 

Signal  base  of  one  character 
of  the  waveform  library  or  of 
one  chip  in  binary  sequences. 

Numl>er 

HJ 

0/J 

Signal  base  for  one  digit  in  a 
nonbinary  system. 

Number 

[1] 

Pm 

Signal  base  for  one  message 
interval. 

Numl>er 

HI 

Ps 

Signal  base  for  one  waveform 
in  one  subchannel  of  a 
MFSK  system. 

Number 

HI 

P\v 

Signal  base  for  one  trans¬ 
mission  word  (of  a  code¬ 
book). 

Number 

HI 

a- 

Elements  in  examples  of  per¬ 
mutation  modulation  (see 
fig.  18). 

Number 

HI 

j 

A  numerical  term  used  in 
Shannon's  theorem  of  othor- 
normnl  expansion. 

Number 

HI 

J(a) 

A  constant  depending  only  on 
the  SNR  and  used  for  the 
design  of  approximate  utility 
curves. 

Number 

HI 

5 

Factor  relating  the  bit  density 
of  a  mathematical  model  to 
Di  by  the  equation  D  61), 

Number 

HI 

*  Neutralized  Unit :  Time  functions  may  he  voltages  or  currents.  To  avoid 
specifying  two  physical  magnitudes,  many  authors  prefer  to  apply  neutralized 
units  with  the  dimension  (watt)1  Any  time  function,  when  expressed  in 
neutralized  units,  will  supply  to  a  one-ohm  resistor  a  power  equal  to  the  square 
of  the  time  function  value,  whether  representing  a  voltage  or  a  current. 
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Symbol 

(Quantity 

Unit 

.MKS 

Dimension 

c 

Energy  contrast  (E  over  X„). 

Number  or  deci¬ 
bel 

HI 

</i 

Energy  contrast  for  one  hit  of 
transmitted  information. 

Number  ordeci- 
l>el 

HI 

Average  energy  contrast  for 
one  character  of  the  wave¬ 
form  library. 

Number  or  deci¬ 
bel 

HI 

Energy  contrast  for  one  digit 
in  a  nonbinary  system. 

Number  or  deci¬ 
bel 

HI 

Error  of  approximation  when 
using  an  expansion  into 
ort honormal  components. 

Number 

HI 

1) 

SNR  difference  between  a 
mathematical  model  and 
Shannon’s  ideal  system. 
Also:  normalized  frequency 
in  the  theory  of  TPW. 

Number  or  deci¬ 
bel 

HI 

VL 

The  normalized  frequency 
establishing  the  practical 
limit  of  the  spectrum  of  a 
TPW. 

Number 

HI 

t) 

1  Fraction  of  the  energy  of  a 
bandlimitcd  waveform  that. 

1  falls  outside  the  band. 

Number 

HI 

Symbol 

Quantity 

Unit 

MKS 

Dimension 

A 

( 'loss-correlation  coefficient, 
between  two  waveforms  in  a 
binary  system. 

Number 

HJ 

Cross-correlation  coefficient 
between  the  i-tli  and  j-th 
waveform. 

Number 

HI 

i-th  symbol  in  a  symbol  alpha- 
bid  in  permutation  modula¬ 
tion. 

Number 

HI 

P 

Characteristic  rate  of  a  system. 

Hertz 

[8-] 

a 

Power  contrast  (SNlt). 

Number  or  deci¬ 
bel 

HI 

Oin 

Input  power  contrast  (input 
SNR). 

Number  or  deci¬ 
bel 

HI 

Oout 

Output  power  contrast  (out¬ 
put  SNR). 

Number  or  deci¬ 
bel 

HI 

<t) 

i-th  orthonormal  waveform 
component  function. 

•Neutralized 

Weight  function  in  the  theory  , 
of  orthogonal  polynomials. 

Number 

HI 

'Neutralized  Unit  :  Time  functions  may  lie  voltages  or  currents.  To  avoid 
specifying  two  physical  magnitudes,  many  authors  prefer  to  apply  neutralized 
units  with  the  dimension  (watt)1/*.  Any  time  function,  when  expressed  in 
neutralized  units,  will  supply  to  a  one-ohm  resistor  a  power  equal  to  the  square 
of  the  time  function  value,  whether  representing  a  voltage  or  a  current. 
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